Machine learning ​

Phil Wang always recreates many cutting edge ML papers with PyTorch. This course & Math for ML seem great. Hugging Face is incredible community & tools. Transformer architectures & Diffusion Models are great. MLU-Explain is a great ML visual explainer.

Looking into using envd together with PyTorch Lightning for my ML experiments.

Lilian's Blog, ML Papers Explained & Understanding Deep Learning are great reads.

Practical Deep Learning for Coders is nice intro course. Andrej Karpathy does great open research on ML.

The Illustrated Transformer & Transformer Recipes are great reads.

XLA is interesting ML compiler.

Notes ​

• A big part of the utility of math (especially in ML) is having breadth rather than depth. The strategy of picking out specific things you don't know from papers and looking them up is only effective if you have the breadth in your background to understand the answers you find.
  • Broad knowledge is also what helps you manage the exponential tree of complexity you're encountering.
    • You won't have seen all the things you come across, but you'll develop the ability to make good judgements about what you need to read to achieve your goals. You'll learn how to recognize when a reference you're reading is more (or less) technical than you need, and how to search for something more appropriate. You'll also learn how and when you can use results without understanding the details.
  • Finally, as a general grad student strategy trying to learn everything just in time is not a path to success. Even if you had the perfect math oracle that you want it would be setting you up to be left behind. All the oracle gives you is the ability to catch up quickly to the ideas of others. Your job as a grad student is to generate new knowledge and to do that you need to seek things out on your own, not just follow along the latest trend. Part of your job is to go out hunting for ideas that your peers haven't found yet and bring them back to your field.
• In supervised learning, you have a bunch of data, a specific question you want to answer, and access to the correct answer to many instances of that question. In unsupervised learning, you have a bunch of data points, and you want to find meaningful patterns in the structure of that data. In reinforcement learning, you have a task you want to take actions to accomplish, and you don't have any access to knowing what the best action is, but after each action you get a rough idea of how good the result was.
• AI doesn't need to follow the human model, just like planes don't need to flap their wings like a bird. For most jobs AI will be very different from humans. Even when AI acts as human for entertainment I would imagine them being very different internally, as their job is to mimic aspects of human behaviors, not actually a human as a whole.
• Almost all of machine learning is about representing data as vectors and performing linear and non-linear transformations in order to perform classification, regression, etc.
• Most of ML is fitting models to data. To fit a model you minimize some error measure as a function of its real valued parameters, e.g. the weights of the connections in a neural network. The algorithms to do the minimization are based on gradient descent, which depends on derivatives, i.e. differential calculus.
• Learn optimization before studying machine learning if you really want to understand what's going on.
• What idiot called it "machine learning" instead of "bias automation".
• If you were to learn only 1 method for explaining machine learning models, it should be Shapley values (SHAP): 1. Model-agnostic: Use with any model. 2. Theoretic foundation: Game theory. 3. Good software ecosystem. 4. Local and global explanations.
• What idiot called it "machine learning" instead of "bias automation".

Links ​

• Neural Networks and Deep Learning book
• Deep Learning Papers Reading Roadmap
• Ask HN: Best way to get started with AI?
• Ask HN: What maths are critical to pursuing ML/AI?
• Ask HN: 'Crash Courses' for Mathematics Related to DL, ML and Data Analysis
• Computational Statistics and Machine Learning Revision Notes
• Stanford CS229 course
• Readings in applied data science
• Learn ML in 3 months
• Deep Learn - Implementation of research papers on Deep Learning+ NLP+ CV in Python using Keras, TensorFlow and Scikit Learn.
• Building Brundage Bot
• Summaries of ML papers
• Code and data for paper "Deep Painterly Harmonization"
• FB AI Tools
• Best Practices for ML Engineering
• Machine Learning From Scratch
• Dive Into ML
• Fermat's Library NIPS comments
• Heroes of Deep Learning: Andrew Ng interviews Ian Goodfellow
• Machine Learning for Humans - Great article.
• Deep Learning for Siri’s Voice: On-device Deep Mixture Density Networks for Hybrid Unit Selection Synthesis
• The Google Brain Team — Looking Back on 2017
• Building the Software 2.0 Stack by Andrej Karpathy from Tesla (2018)
• Deep Learning World
• Machine Learning cheatsheets for Stanford's CS 229
• RAAIS 2018 - François Chollet (Creator of Keras)
• KubeFlow - Machine Learning Toolkit for Kubernetes. (Winding Road to Better Machine Learning Infrastructure Through Tensorflow Extended and Kubeflow)
• KALE (Kubeflow Automated pipeLines Engine) - Aims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows.
• MIT AGI: Deep Learning (Yoshua Bengio) (2018)
• TL-GAN: transparent latent-space GAN - Use supervised learning to illuminate the latent space of GAN for controlled generation and edit.
• Grokking Deep Learning - Repository accompanying "Grokking Deep Learning" book.
• HN: Can we rule out near-term AGI? (2018)
• Introduction to Grenade (Haskell library for Deep Learning)
• Grenade - Deep Learning in Haskell.
• Deep Learning 1: Introduction to Machine Learning Based AI
• Deep Learning cheatsheets for Stanford's CS 230 (2018)
• Deep Learning Book Chapter Summaries - Attempting to make the Deep Learning Book easier to understand.
• PracticalAI - Practical approach to learning machine learning.
• Ask HN: How to incorporate machine learning into day job? (2018)
• RLgraph - Flexible computation graphs for deep reinforcement learning.
• Nevergrad - Gradient-free optimization platform.
• Machine Learning Cheat Sheet
• GANs and Divergence Minimization (2018)
• Convolution arithmetic - Technical report on convolution arithmetic in the context of deep learning.
• FloydHub - Managed cloud platform for data scientists.
• Style Transfer as Optimal Transport - Algorithm that transfers the distribution of visual characteristics, or style, of a reference image onto a subject image via an Optimal Transport plan.
• Looking Back at Google’s Research Efforts in 2018
• Recommenders - Examples and best practices for building recommendation systems, provided as Jupyter notebooks.
• Deep Learning State of the Art (2019) - MIT
• AdaNet - Lightweight TensorFlow-based framework for automatically learning high-quality models with minimal expert intervention.
• DAWNBench - Benchmark suite for end-to-end deep learning training and inference.
• Interpretable Machine Learning (2022) - Guide for Making Black Box Models Explainable. (Code) (2nd edition)
• All You Need to Know About Deep Learning - A kick-starter (2019)
• KubeFlow Pipelines - Machine learning (ML) toolkit that is dedicated to making deployments of ML workflows on Kubernetes simple, portable, and scalable.
• Summary of some ML papers
• Practical Deep Learning for Coders 2019 - (HN) (GitHub)
• Notebooks for the "A walk with fastai2" Study Group and Lecture Series
• The Matrix Calculus You Need For Deep Learning (HN) (HN 2) (Paper)
• Machine Learning Feynman Experience - Collection of concepts I tried to implement using only Python, NumPy and SciPy on Google Colaboratory.
• Tensor2Tensor - Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
• Deep learning drizzle - Various ML, reinforcement learning video lectures. (Code)
• Xfer - Transfer Learning library for Deep Neural Networks.
• List of summer schools in machine learning + related fields
• Most Cited Deep Learning Papers
• Machine Learning CS Columbia Course (2019)
• Learning to Discover Efficient Mathematical Identities - Exploring how machine learning techniques can be applied to the discovery of efficient mathematical identities.
• CleverHans - Adversarial example library for constructing attacks, building defenses, and benchmarking both.
• Google AI Research - Contains code released by Google AI Research.
• Machine Learning Mindmap / Cheatsheet
• Curated list of network embedding techniques
• Deploying Deep Learning - Training guide for inference and deep vision runtime library for NVIDIA DIGITS and Jetson Xavier/TX1/TX2.
• "Adversarial Machine Learning" with Ian Goodfellow (2018)
• HN: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award (2019)
• Large scale K-means and K-nn implementation on NVIDIA GPU / CUDA
• fairseq - Sequence-to-sequence learning toolkit for Torch from Facebook AI Research tailored to Neural Machine Translation (NMT).
• TinyFlow - Tutorial code on how to build your own Deep Learning System in 2k Lines.
• Deep Learning Models - Collection of various deep learning architectures, models, and tips.
• Multi-Level Intermediate Representation Overview - MLIR project aims to define a common intermediate representation (IR) that will unify the infrastructure required to execute high performance machine learning models in TensorFlow and similar ML frameworks. (Talks) (HN) (Slides)
• PySparNN - Approximate Nearest Neighbor Search for Sparse Data in Python.
• Machine Learning Course with Python
• ICML - International Conference on Machine Learning.
• Integrating Domain Knowledge into Deep Learning by Ruslan Salakhutdinov (2019)
• Differentiation for Hackers - The goal of this handbook is to demystify algorithmic differentiation, the tool that underlies modern machine learning.
• Homemade Machine Learning
• Machine Learning Systems are Stuck in a Rut (2019) (HN)
• Literature of Deep Learning for Graphs
• Supplementary Materials for "Unsupervised word embeddings capture latent knowledge from materials science literature", Nature
• Solution of assignment in 2011 Stanford Machine Learning Class
• Autocompletion with deep learning (2019)
• Rules of Machine Learning: Best Practices for ML Engineering
• Python Machine Learning Book (2019) (3rd edition code) (2nd edition code)
• ML and DS Applications in Industry - Curated list of applied machine learning and data science notebooks and libraries across different industries.
• Awesome production machine learning
• Awesome Gradient Boosting Research Papers
• HoloClean - Machine Learning System for Data Enrichment.
• Snorkel - System for quickly generating training data with weak supervision.
• RAdam - On The Variance Of The Adaptive Learning Rate And Beyond.
• Google ML/AI Comic
• Machine Learning Notebooks - Series of Jupyter notebooks that walk you through the fundamentals of Machine Learning and Deep Learning in Python using Scikit-Learn, Keras and TensorFlow 2.
• Streamlit - Fastest way to build custom ML tools. (Web) (Awesome Streamlit) (Streamlit Cheat Sheet) (Tips, tricks, methods, and techniques for building apps with streamlit) (Best of Streamlit) (How to build Streamlit apps on Replit)
• A Gentle Introduction to Bayes’ Theorem for Machine Learning (2019) (HN)
• Practical Deep Learning for Coders (Notes) (Code)
• Part 2: Deep Learning from the Foundations
• Computational Linear Algebra for Coders
• Introduction to Machine Learning for Coders
• Papers with Code - The latest in machine learning. (HN) (GitHub)
• Gradient Descent Derivation (2014)
• Best of Machine Learning Newsletter
• TASO - Tensor Algebra SuperOptimizer for Deep Learning.
• An Exponential Learning Rate Schedule for Deep Learning (2019)
• Billion-scale semi-supervised learning for state-of-the-art image and video classification (2019)
• TRAINS - Auto-Magical Experiment Manager & Version Control for AI.
• Differentiable Optimization-Based Modeling for Machine Learning (2019)
• Notebooks and code for the book "Introduction to Machine Learning with Python"
• Awesome Deep Learning Project Ideas
• Top-down, practical guide to learn AI, Deep learning and Machine Learning
• Most Aesthetically Pleasing ML Research Papers
• Polyaxon - Platform for reproducible and scalable machine learning and deep learning on Kubernetes. (GitHub) (Code)
• Different projects built using fast.ai
• Spell - Fastest and most powerful end-to-end platform for machine learning and deep learning.
• ML portfolio tips (2019)
• DeepMind Research - Contains implementations and illustrative code to accompany DeepMind publications.
• Deep Learning Tutorials
• Prodify - Radically efficient machine teaching. An annotation tool powered by active learning.
• Runway - Professional video editing powered by machine learning — all on the web. (HN)
• An Extended Version Of The Scikit-Learn Cheat Sheet (2014)
• Notes on Machine Learning
• Notes on Deep Learning
• Awesome free deep learning papers
• Teachable Machine - Fast, easy way to create machine learning models for your sites, apps, and more – no expertise or coding required.
• Deep Learning Interview Topics
• Ask HN: Why do so many startups claim machine learning is their long game? (2019)
• End-to-End Machine Learning Courses
• End-to-End Machine Learning Library
• Course material for STAT 479: Machine Learning (FS 2019) taught by Sebastian Raschka at University Wisconsin-Madison
• Clipper - Prediction serving system that sits between user-facing applications and a wide range of commonly used machine learning models and frameworks.
• AI building blocks - from scratch with Python (2018)
• ARUBA: Learning-to-Learn with Less Regret (2019)
• Machine Learning Systems Design
• Guide to Production Level Deep Learning
• Lessons learned building an ML trading system that turned $5k into $200k (2019) (HN)
• Deep Learning for Programmers
• Arcade Learning Environment - Simple object-oriented framework that allows researchers and hobbyists to develop AI agents for Atari 2600 games.
• Space to discuss the future of the ML ecosystem in Rust
• Awesome System for Machine Learning
• Collection of explainer tutorials on how machine learning and statistical concepts work
• 2019’s Top Machine and Deep Learning Research Papers (HN)
• NeurIPS 2019 Schedule
• Machine Learning Crash Course with TensorFlow APIs - Google's fast-paced, practical introduction to machine learning.
• What was your favorite paper of 2019 and why? (2019)
• Ask HN: Full-on machine learning for 2020, what are the best resources?
• Dive into Deep Learning - Interactive deep learning book with code, math, and discussions, based on the NumPy interface. (HN) (Code)
• Resources to learn more about Machine Learning and Artificial Intelligence
• Foundations of Machine Learning book - New edition of a graduate-level machine learning textbook that focuses on the analysis and theory of algorithms.
• Deep Learning book - Resource intended to help students and practitioners enter the field of machine learning in general and deep learning in particular. (Code) (Notes) (Exercises) (LaTeX files for book notation) (PDF) (PDF 2)
• Introduction to Deep Learning - Eugene Charniak - Project-based guide to the basics of deep learning.
• Hands-On Machine Learning with Scikit-Learn, Keras, and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems
• Topological Techniques for Unsupervised Learning (2019)
• Meet AdaMod: a new deep learning optimizer with memory (2020)
• Deep Learning State of the Art (2020)
• The Case for Bayesian Deep Learning (2020) (HN) (Reddit)
• Machine Learning Summer School (2020)
• Machine Learning Summer School videos (2009)
• Turi Create - Simplifies the development of custom machine learning models.
• Private machine learning progress
• Demucs - Code for the paper Music Source Separation in the Waveform Domain.
• Apple at NeurIPS 2019
• Magenta - Make Music and Art Using Machine Learning. (JS Code) (GitHub)
• An overview of gradient descent optimization algorithms (2016)
• What are the current significant trends in ML that are NOT Deep Learning related? (2020)
• Trax - Helps you understand and explore advanced deep learning.
• ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning (Code)
• Cambridge Machine Learning Group
• Convolutional Conditional Neural Processes (2020)
• Privacy Preserving AI (Andrew Trask) (2020)
• Emil’s Story as a Self-Taught AI Researcher (2020)
• Humans of Machine Learning
• Machine Learning Flashcards (HN)
• Awesome Software Engineering for Machine Learning
• Awesome Data Labeling - Curated list of awesome data labeling tools.
• Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020)
• fastai: A Layered API for Deep Learning (2020) (HN)
• The Deep Learning Compiler: A Comprehensive Survey (2020)
• Perceptrons explained
• A Simple Framework for Contrastive Learning of Visual Representations (2020) (Tweet) (Code)
• MediaPipe - Cross-platform framework for building multimodal applied machine learning pipelines.
• ML courses by Zico Kolter
• List of AI Residency Programs
• MIT Introduction to Deep Learning course (2020) (Code) (Videos)
• Stanford Deep Multi-Task and Meta Learning course (2019)
• FastAI book draft (2020) (HN) (Notebooks)
• Pattern Recognition and Machine Learning by Christopher M. Bishop Book (Code) (PDF) (Code/Notes)
• Google Colaboratory - Research project created to help disseminate machine learning education and research.
• Resources for teaching machine learning
• Population Based Augmentation - Algorithm that quickly and efficiently learns data augmentation functions for neural network training.
• NVIDIA Deep Learning Examples for Tensor Cores
• Introduction to Deep Learning and Generative Models course (Code)
• Collection of Conference & School Notes in Machine Learning
• AutoML-Zero - Open source code for the paper: "AutoML-Zero: Evolving Machine Learning Algorithms From Scratch". (HN)
• fastAI course v4
• Ask HN: What is your ML stack like? (2020)
• MLflow - Open source platform to manage the ML lifecycle, including experimentation, reproducibility and deployment. (Code)
• Deep Unsupervised Learning (2020)
• Machine Learning: a Probabilistic Perspective book (Code)
• Introduction to Machine Learning - Carnegie Mellon University (2019)
• Notes on ML courses
• Made With ML - Share what you've Made With ML. (Code) (MLOps Course)
• Backpropagation 101 (2020) - How to trick yourself into understanding backprop without even trying.
• A Spacetime Approach to Generalized Cognitive Reasoning in Multi-scale Learning (2017)
• Open Source Deep Learning Glossary
• Awesome Graph Classification - Collection of important graph embedding, classification and representation learning papers with implementations.
• fast.ai - Making neural nets uncool again. (Code) (Docs) (Course launch) (HN)
• Interactive Tools for ML, DL and Math
• Deep learning with graph-structured representations (2020)
• Interactive Machine Learning Experiments (Web)
• AxCell: Automatic Extraction of Results from Machine Learning Papers (2020)
• Awesome Machine Learning and AI Courses
• How do you keep up with new research? (2020)
• OpenLTH: A Framework for Lottery Tickets and Beyond
• Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning
• Awesome Interpretable Machine Learning
• DLPack: Open In Memory Tensor Structure
• SVM tutorial (HN)
• DeepMind x UCL | Deep Learning Lecture Series (2020)
• Nicholas Carlini: Making and Measuring Progress in Adversarial Machine Learning (2019)
• Deep Learning in Production - Notes and references about deploying deep learning-based models in production.
• Stanford Class on Deep Multi-Task and Meta-Learning (2019) (HN)
• Weights & Biases - Developer tools for ML. Experiment tracking, hyperparameter optimization, model and dataset versioning. (Code) (Docs) (Examples) (Community)
• Protocols and Structures for Inference (PSI) spec - Aims to develop an architecture for presenting machine learning algorithms, their inputs (data) and outputs (predictors) as resource-oriented RESTful web services.
• ML Engineer Roadmap
• Machine Learning for Everyone - In simple words. With real-world examples.
• Deep learning на пальцах (Code)
• How to add AI to your app without knowing anything about AI (2020)
• What I learned from looking at 200 machine learning tools (2020) (HN)
• Interactive Machine Learning List (Code)
• Sema - Live Code Language Design Playground. (Code)
• Model Zoo - Deploy your model with a single line of code. (HN)
• Introduction to Machine Learning with Julia
• Reverb - Efficient and easy-to-use data storage and transport system designed for machine learning research.
• System design patterns for machine learning
• Income strategies to support your ML research (2020)
• Synthetic Data for Deep Learning (2019)
• NeurIPS - Conference on Neural Information Processing Systems.
• Machine Learning Mastery
• Distill - Latest articles about machine learning.
• Compute access for ML training (2020)
• Scann: Scalable Nearest Neighbors (HN)
• CML - Continuous Machine Learning: Bring DevOps to Data Science. (HN) (Code) (Web Code)
• TensorFlow, Keras and deep learning, without a PhD (HN)
• Apple Machine Learning Research (HN)
• Model Card Toolkit - Streamlines and automates generation of Model Cards, machine learning documents that provide context and transparency into a model's development and performance. (Article)
• Top ML Books recommended by experts
• Best practices for performance and cost optimization for machine learning
• Bethge Lab - Perceiving Neural Networks.
• Graphcore code examples
• Machine learning examples and tutorials
• Vintage Factor Analysis with Varimax Performs Statistical Inference (2020)
• An Opinionated Guide to ML Research (2020)
• Amazon's Machine Learning University (HN)
• Progress, Notes, Summaries and a lot of Questions on Machine Learning
• SciML - Open Source Software for Scientific Machine Learning.
• Mihaela Rosca ML notes
• Compression of Deep Learning Models for Text: A Survey (2020)
• The Computational Limits of Deep Learning (2020) (HN)
• ML Ops: Machine Learning Operations (Awesome) (References)
• Benchmarks of approximate nearest neighbor libraries in Python (Web)
• MIT Deep Learning course
• Visualizing Deep Learning (2020)
• Books for Machine Learning, Deep Learning, and related topics
• Graph Representation Learning Book (Reddit)
• Effective testing for machine learning systems (2020) (HN)
• Machine Learning from Scratch (HN) (Code)
• Compose - Machine learning tool for automated prediction engineering. It allows you to structure prediction problems and generate labels for supervised learning.
• Daily scikit-learn tips
• Applied ML - Curated papers, articles, and blogs on data science & machine learning in production.
• Understanding Convolution in Deep Learning (2015)
• 21 Habits I Picked Up While Learning Machine Learning (2019)
• Hacker News for ML
• Think Fast: Tensor Streaming Processor for Accelerating Deep Learning Workloads (2020)
• My Deep Learning Toolchain (2020)
• igel - Machine learning tool that allows you to train/fit, test and use models without writing code. (HN)
• Gradient Boosted Decision Trees (2020) (HN)
• NeurIPS 2020 Accepted Papers
• explained.ai - Deep explanations of machine learning and related topics.
• How to visualize decision trees (2018)
• How to explain gradient boosting (2018)
• Beware Default Random Forest Importances (2018)
• The Mechanics of Machine Learning
• Yann LeCun Spring 2020 DL Course (Videos, Slides, Jupyter Notebooks) (HN) (Article)
• Yann LeCun Deep Learning Course 2021 (HN)
• ML Guide: Feature Store vs Data Warehouse (2020) (HN)
• Grid AI - Seamlessly train hundreds of Machine Learning models on the cloud from your laptop. (HN)
• Getting Started with Applied ML Research (2020)
• Machine Learning Engineering book by Andriy Burkov
• Determined - Deep Learning Training Platform. (Web)
• Phasic Policy Gradient
• Juergen Schmidhuber ML reading list (2015)
• Machine Learning Primer for Interviews
• Confetti AI - Ace Your Machine Learning Interviews.
• Intro to Deep Learning: Use TensorFlow and Keras to build and train neural networks by Kaggle
• Manifold - Model-agnostic visual debugging tool for machine learning. (Web)
• Making With ML YouTube series
• Topological Autoencoders (2020) (Code)
• Awesome Teachable Machine List - Curated list of awesome machine learning projects built with Google's Teachable Machine.
• How to put machine learning models into production (2020)
• Example Machine Learning Scripts for Numerai's Tournament (2020)
• Synthetic Data Vault (SDV) - Synthetic Data Generation for tabular, relational, time series data. (Web)
• Penn Machine Learning Benchmarks - Large, curated repository of benchmark datasets for evaluating supervised machine learning algorithms. (Web)
• 2020 Machine Learning Roadmap
• Responsible Machine Learning - Collection of tools for eXplainable AI (XAI). (Web)
• MI2 DataLab - Group of mathematicians and computer scientists that love to play with data. (GitHub)
• Papers of Robust ML - Mainly focus on defenses.
• Why Deep Learning Works Even Though It Shouldn’t (2020) (Lobsters)
• Some Notable Recent ML Papers and Future Trends (2020)
• Wiki: 2020 ML Interviews Resources & Advice(s)
• Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges (2020)
• Over 200 of the Best Machine Learning, NLP, and Python Tutorials (2018)
• What is a Feature Store? (2020)
• Awesome AutoML Papers - Curated list of automated machine learning papers, articles.
• Machine Learning Systems Design at Stanford course (2020)
• Preferred Networks - Develops practical applications of deep learning and other cutting-edge technologies. (GitHub)
• ML Art - Curated showcase of creative machine learning artworks and projects.
• Decision Making under Uncertainty course (Algorithms for Decision Making book) (Concise Deep Learning Overview (2020) (HN)) (Book Website)
• Effective testing for machine learning systems (2020)
• Reading List for Topics in Multimodal Machine Learning
• Awesome Multimodal Research
• ML and DL related contests, competitions and conference challenges
• ML Visuals - Contains figures and templates which you can reuse and customize to improve your scientific writing.
• DL Visuals - Deep Learning Visuals.
• Deep Learning with Catalyst course
• OpenMined Courses - Learn how privacy technology is changing our world and how you can lead the charge.
• Adversarial ML Threat Matrix
• DeepMind Educational Resources
• Deep Learning (for Audio) with Python (Code)
• Awesome Tensor Compilers
• Adversarial Robustness Toolbox - Python library for Machine Learning Security.
• AI Summer - Learn Deep Learning and Artificial Intelligence. (GitHub)
• How Attention works in Deep Learning (2020)
• Brain Tokyo Workshop - Research materials released by members of the Google Brain team in Tokyo.
• A Novel Framework for Explaining Machine Learning Using Shapley Values (2020) (HN)
• create-ml-app - Template Makefile for ML projects in Python.
• telesto.ai - Competitive marketplace, where you can work on real-life machine learning challenges.
• ML from the Fundamentals - Machine learning in a "from the first principles" style. (Code)
• Interpretability in Machine Learning: An Overview (2020)
• Implicit Rank-Minimizing Autoencoder (2020) (Code)
• ML/CV/NLP study resources
• MIT Mądry Lab - Towards a Principled Science of Deep Learning. (GitHub)
• Scaling Down Deep Learning (2020) (HN) (HN)
• Every Model Learned by Gradient Descent Is Approximately a Kernel Machine (2020) (HN)
• DeepFaceLab - Leading software for creating deepfakes.
• Deep Learning DIY (Code) (GitHub) (Website Code)
• Using JAX to accelerate our research (2020) (HN)
• Stanford MLSys Seminar Series (2020)
• MLCommons - Machine learning innovation to benefit everyone.
• Automated discovery of machine learning optimizations (2020)
• A Visual Tour of Backpropagation (2020)
• Deep Learning GPU Benchmarks (Code)
• What I wish someone had told me about tensor computation libraries (2020) (HN)
• Machine learning could be fundamentally unexplainable (2020)
• Minimum Viable Study Plan for Machine Learning Interviews
• Awesome JAX
• Machine Learning Productivity Hacks (2019)
• Learn Deep Learning: Powerful Mental Models to Accelerate Your Journey (2020) (Tweet)
• OpenNMT - Open source ecosystem for neural machine translation and neural sequence learning. (GitHub)
• How much hyperparameter tuning do you typically end up doing? (2020)
• ZenML - Extensible, open-source MLOps framework for using production-ready Machine Learning pipelines. (Web)
• MIT Parallel Computing and Scientific Machine Learning course (2020)
• Invariant Risk Minimization (2019) (Code)
• Awesome Federated Learning
• Awesome Fraud Detection Research Papers
• Probabilistic Machine Learning: An Introduction (HN)
• Awesome Machine Learning Interpretability
• Reflections on my (Machine Learning) PhD Journey (2020)
• ML beyond Curve Fitting: An Intro to Causal Inference and do-Calculus (2018) (HN)
• Distributed deep learning and inference without sharing raw data (Code)
• Famous ML papers/concepts that are hard to understand (2021)
• MLJAR Automated Machine Learning - Automates Machine Learning Pipeline with Feature Engineering and Hyper-Parameters Tuning. (Web) (HN)
• Best of Machine Learning with Python
• Deep Learning's Most Important Ideas - A Brief Historical Review (2020)
• Awesome Anomaly Detection
• Extending JAX with custom C++ and CUDA code
• Mathematical Engineering of Deep Learning Course (2021)
• Paper List for Style Transfer in Text
• Machine learning with large-scale graphs course (Notes)
• Single-Machine Simulation of Federated Learning Systems (2021)
• Awesome AutoML
• Optimization Methods for Machine Learning and Engineering (2021)
• Three mysteries in deep learning: Ensemble, knowledge distillation, and self-distillation (2021)
• Awesome Federated Computing
• Noah ML Research - Research related code released by Huawei Noah's Ark Lab.
• Prototypical Networks for Few-shot Learning (2017) (Code)
• Domain generalization papers/resources
• The Universal Training Loop of Machine Learning (2021)
• Learning Curve Theory (2021)
• ML Surveys - Survey papers summarizing advances in deep learning, NLP, CV, graphs, reinforcement learning, recommendations, graphs, etc.
• Practical Deep Learning Course
• Diverse Counterfactual Explanations (DiCE) for ML - Generate Diverse Counterfactual Explanations for any machine learning model. (Docs)
• Interpretable Machine Learning - Techniques & resources for training interpretable ML models, explaining ML models, and debugging ML models.
• Awesome Causality - Resources related to causality.
• How to Understand ML Papers Quickly (2021)
• Patterns, Predictions, and Actions Book - A story about machine learning.
• Full Stack Deep Learning Course (2021) (Code)
• Physical Principles for Scalable Neural Recording
• Learn About Transformers: A Recipe (2021)
• An Inferential Perspective on Federated Learning (2021)
• Free Lunch for Few-shot Learning: Distribution Calibration (2020) (Code)
• Enlightening Guide to Machine Learning Interviews
• How Machine Language Works (2021)
• Sliced Score Matching: A Scalable Approach to Density and Score Estimation (2019)
• Accelerating Natural Gradient with Higher-Order Invariance (2018)
• Some interesting observations about machine learning publication practices from an outsider (2021)
• Understanding deep learning requires rethinking generalization (2021) (HN)
• MIT HAN Lab - Accelerate Deep Learning Computing. (GitHub)
• Feature Stores - A Hierarchy of Needs (2021)
• Visualizing Representations: Deep Learning and Human Beings (2015) (Tweet)
• Transformers - Collection of resources to study Transformers in depth.
• Incomplete Deep Learning Guide
• A Year at Google Brain (2020) (HN)
• Pretrained Transformers as Universal Computation Engines (2021) (Code)
• Testing Machine Learning Systems: Code, Data and Models (Tweet)
• Finding Mona Lisa in the Game of Life (2021) (HN)
• Geometric deep learning, from Euclid to drug design (2021) (Reddit)
• GeoGuessing with Deep Learning (2021)
• Awesome Incremental Learning / Lifelong learning
• Out of Distribution Generalization in Machine Learning (2021)
• Label Errors - Label errors in benchmark ML test sets. (Lobsters) (Code)
• What will the major ML research trends be in the 2020s?
• Machine Learning and Deep Learning Courses (2021)
• Awesome Domain Adaptation
• Why machine learning struggles with causality (2021)
• TabNet: Attentive Interpretable Tabular Learning (2020) (Code)
• AutoML.org (GitHub)
• How I built a €25K Machine Learning Rig (2021) (Reddit)
• Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations (2019)
• Machine Learning Collection - Resource for learning about ML, DL, PyTorch and TensorFlow.
• Joint Universal Syntactic and Semantic Parsing (2021)
• A Comprehensive Introduction to Bayesian Deep Learning (2021)
• Stanford Machine Learning with Graphs Course (2021)
• Keepsake - Version control for machine learning. (Web)
• Learning Versatile Neural Architectures by Propagating Network Codes
• Machine learning is going real-time (2020)
• Awesome Normalizing Flows
• Differentiable Model Compression via Pseudo Quantization Noise (2021) (Code)
• The Rise of HuggingFace (2021)
• See through Gradients: Image Batch Recovery via GradInversion (2021) (Reddit)
• Recommendation System using ML and DL
• Mathematical Foundations of Machine Learning (2020)
• How to Write Design Docs for Machine Learning Systems (Code)
• Reproducible Deep Learning (2021) - PhD Course in Data Science. (Code)
• Unsupervised Contrastive Learning of Sound Event Representations (2020) (Code)
• Differentially Private Learning Needs Better Features (or Much More Data) (2021)
• Bias, variance, and their relationship with machine learning algorithms explained (2021)
• Unsupervised Contrastive Learning of Sound Event Representations
• Kobra - Visual programming language for machine learning. (HN)
• Delving into Deep Imbalanced Regression (2021) (Code)
• Comprehensive Survey on Transfer Learning
• Applied Deep Learning Course (Videos)
• Pay Attention to MLPs (2021) (Tweet) (Code)
• E(n) Equivariant Normalizing Flows for Molecule Generation in 3D (2021) (Tweet)
• Fast and Slow Learning of Recurrent Independent Mechanisms (2021) (Tweet)
• Weekly Papers (Daily)
• Sharpness-Aware Minimization for Efficiently Improving Generalization (2020) (Code)
• Apple - Making Mobile Applications Accessible with Machine Learning (2021)
• NYU Deep Learning Spring 2021 (Videos)
• JAX learning resources (2021)
• Machine Learning with Python Cookbook Book (2018)
• Understanding Dimensionality Reduction with UMAP
• Neural Algorithmic Reasoning (2021) (Tweet)
• A Pragmatic Look at Deep Imitation Learning
• Stanford CS229: Machine Learning Course (2021) (Notes)
• Informative Dropout for Robust Representation Learning: A Shape-bias Perspective (2020) (Code)
• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval (2020) (Code)
• Fairification: Making Unfair Programs Fair (2017)
• Fairness and machine learning: Limitations and Opportunities
• Practical Deep Learning for Cloud, Mobile, and Edge (2019)
• Pretrained Encoders are All You Need (2021) (Code)
• The Modern Mathematics of Deep Learning (2021) (HN)
• An Attention Free Transformer (2021) (Code)
• Towards Causal Representation Learning (2021)
• SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training (2021) (Code)
• AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation (2021) (Video)
• Distributed Machine Learning Patterns Book (2021) (Code)
• Revisiting Deep Learning Models for Tabular Data (2021)
• Introduction to Machine Learning Interviews Book (Code)
• Introduction to Machine Learning 2019 at ETH Zürich (Summary)
• Awesome Community Detection Research Papers
• Understanding Deep Learning (2021)
• The Scaling Hypothesis
• ML YouTube Courses
• Neuromatch Academy Deep Learning (NMA-DL) syllabus
• Stanford AI Lab Papers and Talks at CVPR 2021
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption (2021)
• Machine Learning for Beginners - A Curriculum
• You Don’t Need Math For Machine Learning (2021)
• Data Augmentation Resources
• Extremely revealing books that explains everything behind machine learning? (2021)
• ML@B - Machine Learning at Berkeley.
• Awesome Graph Self-Supervised Learning
• Contrastive Representation Learning (2021) (HN)
• Learn Machine Learning Resources
• Popular Machine Learning Interview Questions (2021)
• Solving Machine Learning Performance Anti-Patterns: a Systematic Approach (2021)
• Bayesian learning via stochastic gradient langevin dynamics (2011)
• Megaverse: Simulating Embodied Agents at One Million Experiences per Second (Paper) (Code)
• Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them (2021) (Tweet)
• Linear unit-tests for invariance discovery (2021) (Code)
• In Search of Lost Domain Generalization (2020) (Code)
• Understanding and improving out-of-distribution generalisation with Agnieszka Słowik (2021)
• Algorithmic Concept-based Explainable Reasoning (2021) (Tweet)
• Tsinghua Machine Learning Group (GitHub)
• Zero-Shot Learning Resources
• Machine Learning Collection
• Open MLOps – Open-Source Production Machine Learning (2021) (HN)
• A Gentle Introduction To Gradient Descent Procedure (2021)
• Stanford MLSys Seminars
• Painless Uncertainty for Deep Learning (2021)
• How to avoid machine learning pitfalls: a guide for academic researchers (2021)
• MLOps-Basics
• Machine & Deep Learning Compendium (Code)
• Differentiable Factor Graph Optimization for Learning Smoothers (2021)
• A visual introduction to Gaussian Belief Propagation (2021) (HN)
• Bootstrap your own latent: A new approach to self-supervised Learning (2020) (Code)
• Tutorial: Performance Engineering for Machine Learning and Scientific Computing (2017)
• Must-read papers on Recommender System
• Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts (2018) (Code)
• Machine Learning Bookcamp: Build a portfolio of real-life projects (2021) (Code)
• Product Recommendations (2019)
• Stanford CRFM - Stanford Center for Research on Foundation Models. (GitHub)
• Multi-Task Learning as Multi-Objective Optimization (2019) (Code)
• Hyperparameter Search with spaCy and Weights & Biases (2021) (Tweet)
• Dive into Machine Learning
• The Values Encoded in Machine Learning Research (2021) (Tweet)
• An overview of the theory of overparameterized machine learning (2021)
• Geometric Deep Learning Blueprint (2021)
• Multi-Armed Bandits and Pure-Exploration (2020)
• The First Rule of Machine Learning: Start Without Machine Learning (2021) (HN)
• An Introduction to Weighted Automata in Machine Learning (2021) (Code)
• Ultimate FREE Machine Learning Study Plan
• node2vec: Scalable Feature Learning for Networks (Code)
• Learning to Superoptimize Real-world Programs (2021)
• Data Movement Is All You Need: A Case Study on Optimizing Transformers (2020) (Code)
• Reading List for Topics in Representation Learning
• Курс по машинному обучению для 3 курса факультета ВМК МГУ (2021)
• Deep Learning with Python, Second Edition (2021) (Code)
• Some ML tools (2021)
• Unsolved Problems in ML Safety (2021) (HN)
• Evaluating Predictive Distributions: Does Bayesian Deep Learning Work? (2021) (Code) (Tweet)
• Arsenii Ashukha: Ensemble Generation (2020)
• Imitating Deep Learning Dynamics via Stochastic Differential Equations (2021) (HN)
• What are some ideas that are hyped up in machine learning research but don't actually get used in industry (and vice versa)? (2021)
• Variational Graph Auto-Encoders (2016) (Code)
• Diffusion Normalizing Flow (2021)
• Approximately Correct Machine Intelligence (ACMI) Lab - Research on machine learning, its social impacts, and applications to healthcare. (GitHub) (Twitter)
• Machine Learning, Kolmogorov Complexity, and Squishy Bunnies (2019)
• Fast Machine Learning Lab (GitHub)
• Meaning of interpolation in ML (2021)
• Learning in High Dimension Always Amounts to Extrapolation (2021) (Tweet)
• Awesome Transformer Architecture Search
• Katana ML Skipper - Simple and flexible ML workflow engine. It helps to orchestrate events across a set of microservices and create executable flow to handle requests.
• PaRoT: A Practical Framework for Robust Deep Neural Network Training (2021)
• Superposition of many models into one (2019) (Tweet)
• Meta-Learning Requires Meta-Augmentation (2020)
• Shaking the foundations: delusions in sequence models for interaction and control (2021) (Tweet)
• Introduction to Deep Learning (I2DL) (2021)
• Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data (2021) (Code)
• Advanced-Foundations-of-ML
• Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions (2021) (Code)
• Deep Learning Recommendation Model for Personalization and Recommendation Systems
• MLflow Examples
• 8-bit Optimizers via Block-wise Quantization (2021) (Code)
• Efficiently Modeling Long Sequences with Structured State Spaces (2021) (Code)
• Get started with JAX (Videos)
• How does Jax allocate memory on a TPU? An interactive C++ walkthrough (HN)
• Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method (2021) (Code)
• Introduction to Deep Learning (2021) - 170 Video Lectures from Adaptive Linear Neurons to Zero-shot Classification with Transformers.
• Gradients are Not All You Need (2021) (Tweet) (HN)
• FC2T2: The Fast Continuous Convolutional Taylor Transform with Applications in Vision and Graphics (2021) (Summary)
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingvon (2018) (Code)
• Model compression via distillation and quantization (2019) (Code)
• ML Collective - Independent, nonprofit organization with a mission to make research opportunities accessible and free. (Classics and Trends)
• Awesome MLOps
• Machine Learning Zoomcamp (Code)
• Yann LeCun’s 2021 Deep Learning Course at CDS (2021) (HN)
• Interactive Gradient Descent Demo (2021)
• Why Momentum Really Works (2017)
• HNPE: Leveraging Global Parameters for Neural Posterior Estimation (2021) (Tweet)
• Facebook AI Similarity Search (Faiss) (HN)
• Combined Scaling for Zero-shot Transfer Learning (2021)
• Awesome Distributed Deep Learning
• Neuro Evolution Of Augmented Topologies (2021) (HN)
• machine-config-operator - Managing updates and configuration changes to essentially everything between the kernel and kubelet.
• Factorized Fourier Neural Operators (2021) (Code)
• NeurIPS 2021 Best Paper Awards (2021)
• What areas of deep learning are under-explored? (2021)
• cnvrg.io - Full Stack Machine Learning Operating System.
• Learning with not Enough Data Part 1: Semi-Supervised Learning (2021) (HN)
• Efficient Training of Audio Transformers with Patchout (2021) (Code)
• Machine Learning for Creativity and Design
• Maximum Likelihood Training of Score-Based Diffusion Models (2021) (Code)
• Learning Gradient Fields for Shape Generation (2020) (Code)
• Making Friends with Machine Learning - YouTube
• Schedule - NeurIPS 2021
• The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization (2021) (Code)
• Momentum Residual Neural Networks (2021) (Code)
• Awesome-Zero-Shot-Learning
• Awesome Treasure of Transformers Models Collection
• Self-attention Does Not Need $O(n^2)$ Memory (2021) (Code) (Code)
• Neural Discrete Representation Learning (2021) (Code)
• VSE++: Improving Visual-Semantic Embeddings with Hard Negatives (2018) (Code)
• JAX ResNet - Implementations and checkpoints for ResNet, Wide ResNet, ResNeXt, ResNet-D, and ResNeSt in JAX (Flax).
• Best AI and Deep learning books to read in 2022
• Machine Learning for Combinatorial Optimization - NeurIPS 2021 Competition (Code)
• Tutorial and Summary of Machine Learning
• Never Give Up: Learning Directed Exploration Strategies (2020) (Code)
• A Step Toward Quantifying Independently Reproducible Machine Learning Research (2019) (Tweet)
• ML Hub - Multi-user development platform for machine learning teams. Simple to setup within minutes.
• Adversarial Explainable AI
• AI/ML Tutorials List
• Awesome Diffusion Models
• Autoencoder-based deep metric learning for network intrusion detection (2021) (Code)
• Graph Adversarial Learning Literature
• Trustworthy Machine Learning by Kush R. Varshney
• Recommender System Suits - Open source toolkit for recommender system.
• Randomized Ensembled Double Q-Learning: Learning Fast Without a Model (2021) (Code)
• STATS320: Machine Learning Methods for Neural Data Analysis Course (2021)
• Deep Learning Interviews: Hundreds of fully solved job interview questions from a wide range of key topics in AI (2021) (Code) (HN)
• Diffusion-Models-Seminar
• Awesome Graph Representation Learning
• Machine Learning System Resources (2022)
• Dynamic Tensor Rematerialization (2021) (Review)
• Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021) (Code)
• A Theoretical Framework for Target Propagation (2020) (Code)
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks (2020) (Code)
• Iterative SE(3)-Transformers (2021) (Article)
• Permutation Invariance, DeepSets and Universal Function Approximation
• Review: Deep Learning on Sets
• Contrastive Self-Supervised Learning (2020)
• Top AI Conference Papers with Code
• Awesome Deep Learning papers for industrial Search, Recommendation and Advertising
• Deep Multi-attribute Graph Representation Learning on Protein Structures (2020)
• Variational Diffusion Models (2021) (Code)
• Cheat sheet for the "Deep Learning" course at ETH Zürich
• Convolutional Networks on Graphs for Learning Molecular Fingerprints (2015) (Code)
• Cheatsheet for the Advanced Machine Learning Lecture 2020, ETH Zurich
• Tutorial on amortized optimization for learning to optimize over continuous domains (2022) (Code)
• Growing 3D Artefacts and Functional Machines with Neural Cellular Automata (2021) (Code)
• Generative Flow Networks for Discrete Probabilistic Modeling (2022) (Code)
• Awesome Contrastive Learning
• ML-fairness-gym - Components for building simple simulations that explore the potential long-run impacts of deploying machine learning-based decision systems in social environments.
• MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
• Learning Features with Parameter-Free Layers (2022)
• Gaussian Processes for Machine Learning: Book
• IQ-Learn: Inverse soft-Q Learning for Imitation (2021) (Code)
• On Neural Differential Equations (2022) (Reddit) (Tweet)
• Datamodels: Predicting Predictions from Training Data (2022) (Data)
• Understanding Black-box Predictions via Influence Functions (2020) (PyTorch Code)
• Parameter Prediction for Unseen Deep Architectures (NeurIPS 2021)
• Language-Agnostic Representation Learning of Source Code from Structure and Context (2021) (Code)
• Machine Learning Open Source University
• Exploring hyperparameter meta-loss landscapes with Jax (2021)
• Transformers Can Do Bayesian Inference (2022)
• Norm-based Analysis of Transformer - Implementations for 2 papers introducing to analyze Transformers using vector norms.
• Point-NeRF: Point-based Neural Radiance Fields (2022) (Code)
• ML training compute has been doubling every 6 months since 2010 (HN)
• Using JAX in 2022 (HN)
• Should We Be Using JAX in 2022?
• Parallel Computing and Scientific Machine Learning (SciML): Methods and Applications (Code)
• Awesome Neural ODE - Resources regarding the interplay between differential equations, deep learning, dynamical systems, control and numerical methods.
• LLC: Accurate, Multi-purpose Learnt Low-dimensional Binary Codes (2021) (Code)
• TinyML Paper and Projects
• Multiplicative Filter Networks (2021) (Code)
• Class-incremental learning: survey and performance evaluation on image classification (2020) (Code)
• Appendix: More Is Different In Other Domains (2022)
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition (2022) (Code)
• Nebullvm - All-in-one library for easy-to-use DL compilers.
• SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies (2021) (Code)
• Open Platform for AI (OpenPAI) - Resource scheduling and cluster management for AI.
• Hugging Face Optimum - Accelerate Transformer models for training and inference on targeted hardware. (Tweet)
• Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces (2022) (Tweet)
• What's hot for Machine Learning Research in 2022?
• Machine Learning with PyTorch and Scikit-Learn (2022) (HN) (Author AMA)
• General Cyclical Training of Neural Networks (2022) (Code)
• Machine Learning Interview Questions
• Intro to Continual Learning
• Gradients without Backpropagation (2022) (HN)
• Variational Autoencoders Without the Variation (2022)
• What's your favorite unpopular/forgotten Machine Learning method? (2022)
• Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels (2022) (Tweet)
• Probabilistic Machine Learning: Advanced Topics (Code) (HN)
• AST: Audio Spectrogram Transformer (2021) (Code)
• Practical Machine Learning - Learn by experimenting on state-of-the-art machine learning models and algorithms.
• Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (2022) (Code)
• Optimization for machine learning course (2022) (Code)
• Awesome Long-Tailed Learning
• Offline Reinforcement Learning as One Big Sequence Modeling Problem (2021) (Code)
• Deep Learning’s New Infrastructure (2022)
• Gensyn - Hyperscale, cost-efficient compute protocol for the world's deep learning models. (Twitter)
• Software Engineering for AI/ML -- An Annotated Bibliography
• Physics-Based Deep Learning
• Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation (2019) (Code)
• Tensil - Open-Source ML Accelerators. (HN) (Code)
• Ask HN: What ML platform are you using? (2022)
• Graphsignal - Machine Learning Profiler. (HN)
• Machine Learning and AI with Go
• Everything about Transfer Learning
• Making Deep Learning Go Brrrr From First Principles
• Machine Learning Guide
• Building a ML Transformer in a Spreadsheet (2022)
• Lecture Notes for Machine Learning Theory (2021)
• On Embeddings for Numerical Features in Tabular Deep Learning (2022) (Code)
• Automated Machine Learning in Action Book (2022) (Code)
• ML Course Notes
• The Mathematics of Artificial Intelligence (2022)
• Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision (2021) (Code)
• Fast TreeSHAP: Accelerating SHAP Value Computation for Trees (2021) (Code)
• Memorizing Transformers (2022) (Code) (HN)
• DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents (2022) (Code)
• Metarank - Low-code Machine Learning personalization service. (Code) (HN)
• Research on Tabular Deep Learning
• HuggingFace Blog (Code)
• Learning to Prompt for Continual Learning (2022) (Code)
• Group Equivariant Deep Learning (2022) (Tweet)
• Human-Centered Machine Learning (2022)
• Transformer models: an introduction and catalog — 2022 Edition
• Bayesian Structure Learning with Generative Flow Networks (2022) (Code)
• Neural Networks with Recurrent Generative Feedback (2020) (Code)
• Coursera Machine Learning MOOC by Andrew Ng (Code)
• Chaos is a Ladder: A New Understanding of Contrastive Learning (2022) (Code)
• Efficient-VDVAE: Less is more (2022) (Code)
• Transformer Quality in Linear Time (2022) (Code)
• Machine Learning for Big Code and Naturalness - Survey of Machine Learning for Big Code and Naturalness. (Code)
• Randomized Smoothing of All Shapes and Sizes (2020) (Code)
• Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systems (2021) (Code)
• Deep Maths - machine learning and mathematics (2022)
• Manim Machine Learning - Focused on providing animations and visualizations of common machine learning concepts with the Manim Community Library.
• Anyscale - Effortlessly develop, scale and deploy AI, at any scale. (GitHub)
• Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2021) (Code)
• On the Bottleneck of Graph Neural Networks and its Practical Implications (2021) (Code)
• ML Notebooks - Series of code examples for all sorts of machine learning tasks and applications.
• Awesome Semi-Supervised Learning
• FedScale: Benchmarking Model and System Performance of Federated Learning at Scale (2021) (Code)
• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting (2021) (Code)
• Understand transformer architectures (2022)
• The Future of Machine Learning Tools (2021)
• Transformers Are All You Need: Quick tour through the most popular Neural Net architecture (2022)
• Transformers in Time Series - Curated list of awesome resources (paper, code, data, etc.) on transformers in time series.
• SRBench: A Living Benchmark for Symbolic Regression (2022) (Code)
• How FAANG etc. architect their recommendation systems at scale
• BigScience Research Workshop
• SmallScience - My journey to training a large(ish) transformer model.
• Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks (TSSL-BP) (Code)
• Recommender Systems, Not Just Recommender Models (2022)
• Reading Lists of Machine Learning, Natural Language Processing and etc.
• Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions (2021) (Code) (Code)
• The Principles of Deep Learning Theory (2021) (HN)
• Understanding the Limitations of Variational Mutual Information Estimators (2019) (Code)
• Tensor Puzzles - Solve puzzles. Improve your PyTorch. (Tweet)
• ACM FAccT - 2022 Accepted Papers
• How Attention works, in the field of artificial intelligence
• Awesome Weak-Shot Learning
• Ultimate Awesome Transformer Attention
• What are Diffusion Models? (2022)
• Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization (2020)
• Awesome Conformal Prediction
• The Machine Learning Job Market (2022) (HN)
• Admin-Torch - Understanding the Difficulty of Training Transformers.
• Sampling with Mirrored Stein Operators (2022) (Code) (Tweet)
• data2vec and the future of multimodal learning (2022)
• Mapping Fair ML - Curated list of links and resources for Fair ML and Data Ethics.
• Open Source MLOps - Fuzzy Labs guide to the universe of free and open source MLOps tools.
• Why train when you can optimize? (HN) (Code)
• Data Engineering & Machine Learning Knowledge Hub
• Graph Contrastive Learning with Augmentations (2020) (Code)
• Pseudo Numerical Methods for Diffusion Models on Manifolds (2022) (Code)
• Practical MLOps Book (2021) (Code)
• Disco Diffusion - Notebooks, models and techniques for the generation of AI Art and Animations.
• The Annotated Transformer: Attention is All You Need (Code)
• Metaseq - Codebase for working with Open Pre-trained Transformers.
• Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders (2022) (Code)
• Learning with Noisy Labels
• Pathways: Asynchronous Distributed Dataflow for ML (2022) (Pathways: Google's New ML System)
• Google AI Blog: Alpa: Automated Model-Parallel Deep Learning (2022)
• Compositional Attention: Disentangling Search and Retrieval (2021) (Code) (Code) (Code)
• Awesome Active Learning
• Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022) (Code)
• Introduction to Diffusion Models for Machine Learning (2022) (HN)
• BGU-CS-VIL/DeepDPM: "DeepDPM: Deep Clustering With An Unknown Number of Clusters" (2022) (Code)
• Towards a Learning-Based Query Optimizer (2022) (HN)
• Scaling-up Diverse Orthogonal Convolutional Networks with a Paraunitary Framework (2021)
• Data Distributional Properties Drive Emergent Few-Shot Learning in Transformers (2022) (Tweet)
• Machine Learning Specialization - DeepLearning.AI (HN)
• MLU-Explain - Visual explanations of core machine learning concepts. (HN)
• Planning with Diffusion for Flexible Behavior Synthesis (2022) (Code)
• Solutions to Recommender Systems competitions
• Recipe for a General, Powerful, Scalable Graph Transformer (2022) (Code)
• How to properly handle hyperparameter configs in ML repos (2022)
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022) (Code)
• Machine Learning Design patterns - Software Architecture for ML engineers.
• Hopular: Modern Hopfield Networks for Tabular Data (2022) (Code)
• Some thoughts on machine learning with small data (2022) (HN)
• Most elegant/beautiful ideas in ML? (2022)
• Apple Silicon DL benchmarks - Collection of ML scripts to test the M1 Pro MacBook Pro.
• Post-Modern ML Stack
• Improving Discrete Latent Representations With Differentiable Approximation Bridges (2019) (Code)
• Semantic Search and GIFs (HN)
• Materials for workshops on the Hugging Face ecosystem
• Deploying Transformers on the Apple Neural Engine (2022) (HN)
• A comprehensive review of Binary Neural Network (2022)
• Self-organising Systems from Google
• Improved Denoising Diffusion Probabilistic Models (2021) (Code)
• What topics to learn to get 'cutting edge AI' (2022)
• Probability flow solution of the Fokker-Planck equation (2022)
• Review of latest Score Based Generative Modeling papers - All diffusion papers reverse chronological.
• Intro resources on diffusion/score-matching models (2022)
• Meta Optimal Transport (2022) (Code)
• envd - Development environment for machine learning.
• Awesome Open Source MLOps
• Generalised Implicit Neural Representations (2022) (Code)
• Diffusers - Provides pretrained diffusion models across multiple modalities, such as vision and audio, and serves as a modular toolbox for inference and training of diffusion models. (Fork)
• Lightning AI - Use Lightning Apps to build everything from production-ready, multi-cloud ML systems to simple research demos.
• Latent World Models For Intrinsically Motivated Exploration (2020) (Code)
• Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt (2022) (Code)
• Self-Supervised Learning from Images: Up-to-date reading list
• How robust are pre-trained models to distribution shift? (2022)
• MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022) (Code) (Web)
• MLOPs Primer - Collection of resources to learn about MLOPs.
• Are wider nets better given the same number of parameters? (2020) (Code)
• Brandon Amos's presentation slides
• Ethical Principles for Web Machine Learning (Code)
• MLC - Machine Learning Compiler.
• Golan Levin's lectures
• RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network (2022) (Code)
• Pen and paper exercises in machine learning (Paper) (HN)
• Dual Curriculum Design - Implementations of robust Dual Curriculum Design (DCD) algorithms for unsupervised environment design.
• gDDIM: Generalized denoising diffusion implicit models (2022) (Code)
• Awesome Active Learning
• Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications (Code)
• The Berkeley Crossword Solver
• MLC - Machine Learning Compiler.
• Recommender System on MovieLens dataset
• Awesome Radar Perception - Curated list of radar datasets, detection, tracking and fusion.
• MLGO: A Machine Learning Framework for Compiler Optimization (2022) (HN)
• Supporting GPU-accelerated Machine Learning with Kubernetes and Nix (2022)
• Transfer Learning with Deep Tabular Models (2022) (Code)
• Speech Denoising in the Waveform Domain with Self-Attention (2022) (Code)
• SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data (2021) (Code)
• MIT: Deep Learning for Art, Aesthetics, and Creativity (2022)
• 500 AI Machine learning Deep learning Computer vision NLP Projects with code
• Deep Learning Curriculum
• Diffusion models - Minimal standalone example of diffusion model.
• Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021) (Code)
• ML code generation vs. coding by hand: what we think programming will look like (2022) (HN)
• Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning (2021) (Code)
• Recommender Systems course at Polimi
• A General Recipe for Likelihood-free Bayesian Optimization (2022) (Code)
• Generative Coarse-Graining of Molecular Conformations (2022) (Code)
• Thoughts on ML Engineering After a Year of My PhD (2022) (HN)
• Towards a General Purpose CNN for Long Range Dependencies in ND (2022) (Code)
• Formal Algorithms for Transformers (2022) (HN)
• Practical Deep Learning for Coders 2022 (HN)
• ML is not that good at predicting consumers' choices (2022) (HN)
• TabTransformer: Tabular Data Modeling Using Contextual Embeddings (2020) (Code)
• Automatic Symmetry Discovery with Lie Algebra Convolutional Network (2021) (Code)
• Awesome Time Series Papers
• Deep Learning setup made easy with EC2 Remote Runner and Habana Gaudi (2022)
• Efficient Deep Learning Book (Code)
• DALLE2 LAION - Collection of resources and tools for LAION's pre-trained DALLE-2 model.
• Awesome Novel Class Discovery
• Harvard ML Course (2019)
• Transformers as Meta-Learners for Implicit Neural Representations (2022) (Code)
• AdaCat: Adaptive Categorical Discretization for Autoregressive Models (2022) (Code)
• Awesome Decision Tree Research Papers
• Why do tree-based models still outperform deep learning on tabular data? (2022) (HN)
• Improving Sample Efficiency in Model-Free Reinforcement Learning from Images (2020) (Code)
• Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models (2021) (Code)
• First Italian School on Geometric Deep Learning (2022)
• DeepTIMe: Deep Time-Index Meta-Learning for Non-Stationary Time-Series Forecasting (2022) (Code)
• Stable Diffusion Akashic Records - Compendium of informations regarding Stable Diffusion (SD).
• Ask HN: In 2022, what is the proper way to get into machine/deep learning? (2022)
• 1 week of Stable Diffusion (HN)
• What are Diffusion Models? (2021) (Lobsters)
• Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models (2022) (Code)
• Awesome AI image synthesis
• First-Time Machine Learning Playbook (2022)
• Grokking Stable Diffusion
• Notes for prompt engineering
• Course content and resources for the AIAIART course
• Stable Diffusion UI
• Score-Based Generative Modeling with Critically-Damped Langevin Diffusion (2022) (Code)
• Fast Sampling of Diffusion Models with Exponential Integrator (2022) (Code)
• Frozen️ in Time - Joint Video and Image Encoder for End-to-End Retrieval.
• A Short Chronology Of Deep Learning For Tabular Data (2022) (HN)
• Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer (2022) (Tweet)
• Stat.ML Papers Twitter - Unofficial updates of statistical machine learning papers on arXiv.
• Machine Learning authors/titles recent submissions
• Are Transformers Effective for Time Series Forecasting? (2022) (Code)
• Motiflets -- Fast and Accurate Detection of Motifs in Time Series (2022)
• Exploring Differential Geometry in Neural Implicits
• REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfer (2022) (Code)
• The AI Epiphany - YouTube
• A Review of Sparse Expert Models in Deep Learning (2022)
• Conformal Prediction - Lightweight, useful implementation of conformal prediction on real data.
• Collection of papers on divergence and quality diversity
• TACTiS: Transformer-Attentional Copulas for Time Series (2022) (Code)
• Deep Learning Systems: Algorithms and Implementation
• Git Re-Basin: Merging Models modulo Permutation Symmetries (2022) (Code)
• Awesome-Machine-Unlearning
• sample-generator - Tools to train a generative model on arbitrary audio samples.
• Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions (2022)
• Start Here with Machine Learning
• ZeroC: A Neuro-Symbolic Model for Zero-shot Concept Recognition and Acquisition at Inference Time (2022)
• Learning to Accelerate Partial Differential Equations via Latent Global Evolution (2022)
• Intro to Transformers (2022) (Tweet)
• Maximum Likelihood Training of Implicit Nonlinear Diffusion Models (2022) (Code)
• On the Paradox of Learning to Reason from Data (2022)
• Mega: Moving Average Equipped Gated Attention (2022) (Code)
• Toy Models of Superposition (2022) (Tweet)
• Learning to Learn with Generative Models of Neural Network Checkpoints (2022) (Code) (Web) (Tweet)
• Didact AI: The anatomy of an ML-powered stock picking engine (2022)
• Deep Learning Examples
• Variational Inference for Infinitely Deep Neural Networks (2022) (Code)
• First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization (2022) (Code)
• Gradient Gating for Deep Multi-Rate Learning on Graphs (2022) (Code)
• Collective Intelligence for Deep Learning: A Survey of Recent Developments (2022)
• TT-NF: Tensor Train Neural Fields (2022) (Code)
• Forecasting Future World Events with Neural Networks (2022) (Code)
• NeuML - Applying machine learning to solve everyday problems. (GitHub)
• ML founder starter stack (2022)
• Build AI product MVP quickly with HuggingFace (2022)
• Efficient and Scalable Parallel Functional Programming Through Disentanglement (2022) (Tweet)
• What's New In Machine Learning? (2022)
• Shervine Amidi's ML Cheat Sheets
• Complete Machine Learning Package - Comprehensive machine learning repository containing 30+ notebooks on different concepts, algorithms and techniques.
• TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second (2022) (Code) (Summary) (HN)
• Matérn Gaussian Processes on Graphs (2021) (Code)
• How to model and predict churn using deep learning (2016) (Tweet)
• Poisson Flow Generative Models (2022) (Code)
• Latent Space Smoothing for Individually Fair Representations (2021) (Tweet)
• Volodymyr Kuleshov - YouTube
• VectorAdam for Rotation Equivariant Geometry Optimization (2022) (Code)
• Learning Fast and Slow for Online Time Series Forecasting (2022) (Code)
• What Makes Convolutional Models Great on Long Sequence Modeling? (2022) (Code)
• Machine Learning Specialization by Andrew Ng notes (2022)
• Deep Generalized Schrödinger Bridge (2022) (Code)
• Awesome Full Stack Machine Learning Engineering Courses
• Monolith: Real Time Recommendation System With Collisionless Embedding Table (2022) (Tweet)
• GENIE: Higher-Order Denoising Diffusion Solvers (2022) (Web)
• PDEBENCH: An Extensive Benchmark for Scientific Machine Learning (2022) (Code)
• Broken Neural Scaling Laws (2022) (Code)
• Failed Machine Learning - Compilation of high-profile real-world examples of failed machine learning projects.
• Thoughts on my first machine learning project (2022)
• Are You A Cat? ML learning project
• The gap between theory and practice in function approximation with deep neural networks (2020) (Code)
• Bayesian neural network papers
• Understanding Bayesian Deep Learning
• Disentangled Image Colorization via Global Anchors
• Estimating High Order Gradients of the Data Distribution by Denoising (2021) (Code)
• Sharpness-Aware Minimization for Efficiently Improving Generalization (2020) (Code)
• Closed-form Continuous-time Neural Models (2021) (Code)
• Creative AI
• ZenBytes - Simple guide to MLOps through ZenML and its various integrations.
• Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting (2022) (Code)
• Deep Image Processing
• Uni-Perceiver models
• Compiler Provenance Recovery for Multi-CPU Architectures Using a Centrifuge Mechanism (2022)
• Efficient AI Backbones - Including GhostNet, TNT (Transformer in Transformer), AugViT, WaveMLP and ViG developed by Huawei Noah's Ark Lab.
• Some Math behind Neural Tangent Kernel (2022)
• Evolution through Large Models (2022) (Code)
• Causal Confounds in Sequential Decision Making (2022)
• MIP Workshop 2023 Computational Competition
• Statistical vs. Deep Learning forecasting methods (HN)
• USB: A Unified Semi-supervised Learning Benchmark for Classification (2022) (Code)
• Walk with fastai - the missing pieces for success (Code)
• Awesome NVIDIA Isaac Gym - Curated list of awesome NVIDIA Issac Gym frameworks, papers, software, and resources.
• Deep Learning Fundamentals - Lightning AI
• Transformers for software engineers (2022)
• A Generalist Neural Algorithmic Learner (2022)
• AlphaCode Attention Visualization (HN)
• The Illustrated Transformer (2018)
• Ask HN: How to get back into AI? (2022)
• Semi-Discrete Normalizing Flows through Differentiable Tessellation (2022) (Code)
• Learning the Travelling Salesperson Problem Requires Rethinking Generalization (2021) (Code)
• Learn Machine Learning in 3 Months (PyTorch Curriculum)
• Transformers for software engineers (2022)
• Measuring Data (2022) (Tweet)
• Learning to generate line drawings that convey geometry and semantics (2022) (Code)
• Data2vec 2.0: Highly efficient self-supervised learning for vision, speech and text (2022)
• X-modaler - Versatile and high-performance codebase for cross-modal analytics.
• Normconf - Tech conference about all the stuff that matters in data and machine learning. (Awesome) (Twitter) (Videos)
• Machine Learning for Engineers Course - YouTube (2022)
• Uncertain: Modern Topics in Uncertainty Quantification
• Learn ML yourself - resources (2020)
• MEMIT: Mass-Editing Memory in a Transformer - Editing thousands of facts into a transformer memory at once.
• Lightning fast recommendation with Birdland
• Pythia: Interpreting Autoregressive Transformers Across Time and Scale
• Thinking like Transformers (HN) (Code)
• OLLA: Optimizing the Lifetime and Location of Arrays to Reduce the Memory Usage of Neural Networks (2022) (Code)
• Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoning (2022) (Code)
• Extremely Simple Activation Shaping for Out-of-Distribution Detection (2022) (Code)
• Nice personal ML/DS blogs
• Ask HN: Which are your favorite machine learning blogs? (2022)
• Liquid Structural State-Space Models (2022) (Code)
• OpenXLA - Community-driven and modular open source compiler for ML.
• Graph-based Fraud Detection Papers and Resources
• Trustworthy AI related projects
• Paper List for In-context Learning
• Awesome Domain Generalization
• Editing Models with Task Arithmetic (2022) (Code)
• POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging (2022) (Code)
• Amortized Inference for Causal Structure Learning (2022) (Code)
• Awesome Machine Learning for Combinatorial Optimization Resources
• Magick - Visual IDE for no-code data pipelines and multimodal agents.
• Rigging the Lottery: Making All Tickets Winners (2019) (Code)
• Ask HN: What are the foundational texts for learning about AI/ML/NN? (2023)
• ML Papers of The Week
• Efficient Deep Learning - Collection of recent methods on (deep) neural network compression and acceleration.
• Geoffrey Hinton publishes new deep learning algorithm (2023) (HN)
• ML Papers Explained - Explanations to key concepts in ML.
• Tracr: Compiled Transformers as a Laboratory for Interpretability (2023) (Code)
• Understanding Deep Learning Book (2022) (Code)
• Riemannian Score-Based Generative Modelling (2022) (Code)
• Techniques to improve reliability of LLMs (HN)
• Learning Compressed Transforms with Low Displacement Rank (2018) (Code)
• Ask HN: ML Papers to Implement (2023)
• Hungry Hungry Hippos: Towards Language Modeling with State Space Models (2022) (Code)
• Intuitive Tutorial to Gaussian Processes Regression
• MPCFormer: fast, performant and private Transformer inference with MPC (2022) (Code)
• Deep Equilibrium Approaches to Diffusion Models (2022) (Code)
• Causal Transformer Decoder
• Illustrated Machine Learning (Code)
• The Transformer Family (2023) (HN)
• Just know stuff (or, how to achieve success in a machine learning PhD) (2023) (HN)
• Deep invariant networks with differentiable augmentation layers (2022) (Code)
• Weakly-supervised Anomaly Detection: A Survey
• Awesome Federated Machine Learning
• Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale (2022)
• Impactful and widely cited papers and literature on ML
• Resources of deep learning for mathematical reasoning
• Homomorphic Learning study
• Speculative Sampling (2023)
• Numerically Stable Softmax and Cross Entropy (2022)
• An Intuition for Attention (2022)
• Latent State Marginalization as a Low-cost Approach for Improving Exploration (2023) (Code)
• The Little Learner (2023) - Highly accessible, step-by-step introduction to deep learning, written in an engaging, question-and-answer style. (HN)
• Paper list of RGBD semantic segmentation
• Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations (2022) (Code)
• Auxiliary Learning as an Asymmetric Bargaining Game (2023) (Code)
• Block-Recurrent Transformers (2022) (Code)
• Awesome Causality Algorithms
• Transformer learning explained: Coinductive guide to inductive transformer heads (2023) (HN)
• MultiViz: Towards Visualizing and Understanding Multimodal Models (Code) (Paper)
• Awesome Transformers - Curated list of awesome transformer models.
• Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models (2022) (Code)
• Dropout Reduces Underfitting (2023) (Code)
• Consistency Models (2023) (Code) (Code) (Code)
• Online active learning in 80 lines of Python (2023)
• Prismer: A Vision-Language Model with An Ensemble of Experts (2023) (Code)
• Compose & Embellish: Well-Structured Piano Performance Generation via A Two-Stage Approach (2022) (Code)
• Clifford Layers - Neural Network layers inspired by Clifford / Geometric Algebras.
• GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers (2022) (Code)
• Precise Zero-Shot Dense Retrieval without Relevance Labels (2022) (Code)
• Superposition, Memorization, and Double Descent (2023)
• Teaching material for Causal ML
• Symbolic Execution Papers
• Programming Machine Learning: From Coding to Deep Learning - Elixir Livebooks
• Introduction to Autoencoders (2023)
• Awesome Uncertainty Deep Learning
• RFdiffusion - Open source method for structure generation, with or without conditional information.
• Advances in Foundation Models Course (2023)
• Hello Deep Learning (2023)
• AI Alignment Forum
• Infinity-Diff: Infinite Resolution Diffusion with Subsampled Mollified States (2023) (Code)
• Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Sampling (2021) (Code)
• Convolutions for Sequence Modeling
• What Can Transformers Learn In-Context? A Case Study of Simple Function Classes (2022)
• TRAK: Attributing Model Behavior at Scale (2023) (Code)
• Building Machine Learning Apps with Hugging Face: LLMs to Diffusion Modeling (2023)
• On Efficient Training of Large-Scale Deep Learning Models: A Literature Review (2023)
• Example models using DeepSpeed
• Transformer Deep Dive: Parameter Counting
• Automatic Gradient Descent
• Image Restoration with Mean-Reverting Stochastic Differential Equations (2023) (Code)
• Contriever: Unsupervised Dense Information Retrieval with Contrastive Learning
• Transformer Math 101 (2023)
• Exploratory Analysis of TRLX RLHF Transformers with TransformerLens (2023)
• EleutherAI Second Retrospective: The long version (2023)
• The Big Dictionary of MLOps
• Learning to Substitute Ingredients in Recipes (2023) (Code)
• Mass Editing Memory in a Transformer (2022)
• Bridging Discrete and Backpropagation: Straight-Through and Beyond (2023) (Code)
• A Cookbook of Self-Supervised Learning (2023) (HN)
• Table Transformer: Deep learning model for extracting tables from unstructured documents (HN)
• What Is ML Compilation
• The Little Book of Deep Learning (HN)
• Fun and Hackable Tensors in Rust, From Scratch (2023) (Reddit)
• Efficient encoder-decoder architecture with top-down attention for speech separation
• ImageBind - One Embedding Space To Bind Them All.
• COLA: Contextualized Commonsense Causal Reasoning from the Causal Inference Perspective (2023)
• Caikit - AI toolkit that enables users to manage models through a set of developer friendly APIs.
• SAITS: Self-attention-based imputation for time series (2023) (Code)
• SoundStorm: Efficient Parallel Audio Generation (2023) (Code)
• OpenFE - Automated feature generation with expert-level performance.
• MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers (2023) (Code)
• QLoRA: Efficient Finetuning of Quantized LLMs (2023) (Code)
• PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction (2023) (Code)
• What Machine Learning Tells Us About the Mathematical Structures of Concepts (2023)
• Visuals and Machine Learning
• Ask HN: What are some of the best papers on AI, ML, DL and their applications? (2023)
• ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation (2023) (Code)
• Yann LeCun, Chief AI Scientist at Meta AI: From Machine Learning to Autonomous Intelligence (2023)
• Minimizing Trajectory Curvature of ODE-based Generative Models (2023) (Code)
• Cross-Modal Fine-Tuning: Align then Refine (2023) (Code)
• NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
• On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline (2022) (Code)
• A Mechanistic Interpretability Analysis of Grokking (2022)
• Tool Learning Papers - Must-read papers on tool learning with foundation models.
• Awesome Graph Causal Learning
• Brainformers: Trading Simplicity for Efficiency (2023) (HN)
• CodeCompose: A Large-Scale Industrial Deployment of AI-assisted Code Authoring (2023) (HN)
• Learning Transformer Programs (2023) (Code)
• Learning Neural Parametric Head Models (2023) (Code)
• GGML - AI at the Edge. (HN)
• Transformer Inference Arithmetic (2022)
• Context is all you need - Multimodal vector search with personalization (2023)
• U-Net CNN in APL: Exploring Zero-Framework, Zero-Library Machine Learning (2023) (HN)
• LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022) (HN)
• OptiML: Optimizing Efficiency in Machine Learning (2022)
• Any Deep ReLU Network is Shallow (2023) (HN)
• The Little Book of Deep Learning (HN)
• The Self-Supervised Learning Cookbook (2023)
• Building Autograd Engine & Neural Network Library: An Interactive Guide
• Petaflops to the People — with George Hotz of tinycorp (2023)
• Bayesian Learning
• Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics (2023) (Code)
• DreamEdit: Subject-driven Image Editing (Code)
• What is currently the best theoretical book (or notes) about Convolutional Neural Networks? (2023)
• Sparse Bayesian Optimization (2022) (Code)
• Various machine learning algorithms. Mostly vanilla.
• Machine Learning in C (2023)
• Efficient Deep Learning Systems Course
• emlearn - Machine Learning inference engine for Microcontrollers and Embedded devices.
• A Theory on Adam Instability in Large-Scale Machine Learning (2023) (HN)
• Attention Is Off By One (2023) (Lobsters) (HN)
• Upgini - Data search & enrichment library for Machine Learning.
• Gzip and nearest neighbors for text classification
• TabR: Unlocking the Power of Retrieval-Augmented Tabular Deep Learning (2023) (Code)
• Getting Started with Torch-Harmonics
• Aligned Diffusion Schrödinger Bridges (2023) (Code)
• SpaceTime - Neural net architecture for time series.
• Methods and Implements of Deep Clustering
• Neural Priming for Sample-Efficient Adaptation (2023) (Code)
• Do Machine Learning Models Memorize or Generalize? (2023) (HN)
• Is everything just transformers now? (2023)
• Deep Learning Systems Course (2023) (HN)
• The Little Book of Deep Learning (2023)
• Automatically classifying the content of sound files using ML (2023)
• Bayesian Flow Networks (2023) (HN) (Code)
• Ask HN: Is making a self taught transition to AI/ML related fields possible? (2023)
• Jürgen Schmidhuber podcast interview (2023)
• Scaling MLPs: A Tale of Inductive Bias (2023) (Code)
• Composable Function-preserving Expansions for Transformer Architectures (2023) (HN)
• “Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors (2023) (Code)
• AutoML Web App - Web App for training Machine Learning pipelines with MLJAR AutoML.
• PLANTAIN - Fast and accurate molecular docking with an AI pose scoring function.
• Learning AI - Notes and exploration code for learning about AI/ML.
• Awesome Temporal Action Localization
• Google Labs Prototyping Project