ML Models ​

Tangram, PostgresML, ZenML & Merlin seem neat. Looking into using Cog to package ML models.

Currently use Modal or Banana to serve ML models (mostly generative) with their HTTP template. Replicate is great too. Mosec is great tool.

XManager is nice for managing ML model experiments.

Web AI or Transformers.js are great for running models in browser.

Links ​

• Lip Reading - Cross Audio-Visual Recognition using 3D Architectures
• Cortex - API platform for machine learning engineers. (Web)
• BentoML - Model Serving Made Easy. (Docs)
• Lobe - Helps you train machine learning models with a free, easy to use tool. (Tweet) (HN)
• Algorithmia - Deploy Autoscaling ML Models using Serverless Microservices. (GitHub)
• How to Deploy ML models with AWS Lambda (2020)
• Verta - MLOps software supports model development, deployment, operations, monitoring.
• Guild AI - Experiment tracking, ML developer tools. (Code)
• Neuralet - Open-source platform for edge deep learning models on GPU, TPU, and more. (Code)
• InterpretML - Fit interpretable models. Explain blackbox machine learning.
• What-If Tool - Visually probe the behavior of trained machine learning models, with minimal coding. (Code)
• LightAutoML - Automatic model creation framework.
• Evidently - Interactive reports to analyze machine learning models during validation or production monitoring. (Web)
• MLCube - Project that reduces friction for machine learning by ensuring that models are easily portable and reproducible. (Docs)
• Service Streamer - Boosting your Web Services of Deep Learning Applications.
• Shapash - Makes Machine Learning models transparent and understandable by everyone. (Web) (HN)
• BudgetML: Deploy ML models on a budget (HN)
• Introducing Model Search: An Open Source Platform for Finding Optimal ML Models (2021)
• Model Search - Framework that implements AutoML algorithms for model architecture search at scale.
• Embedding stores (2021)
• Running ML models in a game (and in Wasm!) (2020)
• Deep learning model compression (2021)
• ModelDB - Open Source ML Model Versioning, Metadata, and Experiment Management.
• Gradio - Generate an easy-to-use UI for your ML model, function, or API with only a few lines of code. (Code)
• Awesome Model Quantization
• Tracking the Performance of Your Machine Learning Models With MLflow (2021)
• Counterfit - CLI that provides a generic automation layer for assessing the security of ML models.
• Convect - Instant Serverless Deployment of ML Models. (HN)
• Using Argo to Train Predictive Models (2021) (HN)
• Yellowbrick - Visual analysis and diagnostic tools to facilitate machine learning model selection. (Docs)
• Deep Learning Model Convertors
• Tuning Model Performance (2021)
• SHAP - Game theoretic approach to explain the output of any machine learning model.
• Lazy Predict - Helps build a lot of basic models without much code and helps understand which models works better without any parameter tuning.
• How to Monitor Models (2020)
• How to Serve Models (2020)
• StudioML - Python model management framework. (Code)
• MLapp - ML model serving app based on APIs.
• Machine Learning Hyperparameter Optimization with Argo (2021)
• Snakepit - Coqui's machine learning job scheduler.
• MLServer - Inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more. (Docs)
• SpotML - Managed ML Training on Cheap AWS/GCP Spot Instances. (HN)
• Mosaic ML - Making ML Training Efficient. (Tweet) (Intro)
• RecoEdge - Deploy recommendation engines with Edge Computing.
• MLRun - Open-Source MLOps Orchestration Framework.
• PrimeHub - Toil-free multi-tenancy machine learning platform in your Kubernetes cluster. (Docs)
• MLeap - Deploy ML Pipelines to Production. (Docs)
• ServingMLFastCelery - Working example for serving a ML model using FastAPI and Celery.
• Cog - Containers for machine learning. (HN) (Tweet)
• Explaining Machine Learning Models: A Non-Technical Guide to Interpreting SHAP Analyses (2021)
• Improving a Machine Learning System Is Hard (2021)
• Removal-based explanations - Lightweight implementation of removal-based explanations for ML models.
• Gordo - Building thousands of models with timeseries data to monitor systems.
• Mosec - Model Serving made Efficient in the Cloud.
• MLNotify - Add just 1 import line and MLNotify will let you know the second it's done.
• Build models like we build open-source software (2021) (HN)
• Deepchecks - Python package for comprehensively validating your machine learning models and data with minimal effort.
• Auptimizer - Automatic ML model optimization tool.
• runx - Deep Learning Experiment Management.
• ML Console - Web app to train ML models, for free and client-side. (HN)
• MMDeploy - OpenMMLab Model Deployment Framework. (Docs)
• Wonnx - Aimed at being an ONNX Provider for every GPU on all platforms written in 100% Rust.
• How to Build a Machine Learning Demo in 2022
• Zetane Viewer - ML models and internal tensors 3D visualizer.
• ONNX Model Zoo - Collection of pre-trained, state-of-the-art models in the ONNX format.
• Model Zoo for MindSpore
• Seldon - Machine Learning Deployment for Kubernetes. (GitHub)
• ORMB - Docker for Your ML/DL Models Based on OCI Artifacts.
• Spaces - Hugging Face (Tweet)
• Nanit’s AI Development Process (2022)
• ailia SDK ML Models
• BentoML - Simplify Model Deployment. (GitHub)
• bentoctl - Fast model deployment with BentoML on cloud platforms.
• ModelCenter - Efficient, Low-Resource, Distributed transformer implementation based on BMTrain.
• PostgresML - End-to-end machine learning system. It enables you to train models and make online predictions using only SQL, without your data ever leaving your favorite database. (Web) (HN)
• UniLM AI - Pre-trained models across tasks (understanding, generation and translation), languages, and modalities.
• Domino - Discover slices of data on which your models underperform.
• Merlin - Kubernetes-friendly ML model management, deployment, and serving.
• Baseten - Build ML-powered applications. (HN)
• Triton Inference Server - Provides a cloud and edge inferencing solution optimized for both CPUs and GPUs.
• Feature Store - Feature store co-designed with a data platform and MLOps framework. (Announcement)
• Auto-ViML - Automatically Build Variant Interpretable ML models fast.
• Angel - Flexible and Powerful Parameter Server for large-scale machine learning.
• Trainer - General purpose model trainer, as flexible as it gets.
• onnxcustom - Tutorial on how to convert machine learned models into ONNX.
• Vetiver - Version, share, deploy, and monitor models.
• Cloud TPU VMs are generally available (2022) (HN)
• NannyML - Detecting silent model failure.
• Pydra - Pydantic and Hydra for configuration management of model training experiments (2022)
• BlindAI - Confidential AI inference server.
• Vertigo - AI for IoT & The Edge.
• Compair - Model evaluation utilities.
• LightAutoML - Fast and customizable framework for automatic ML model creation (AutoML).
• MLEM - Version and deploy your ML models following GitOps principles. (Web)
• Serving ML at the Speed of Rust (2022) (HN)
• Sematic - Open-source framework to build ML pipelines faster. (Web) (HN)
• ML Platform Workshop - Example code for a basic ML Platform based on Pulumi, FastAPI, DVC, MLFlow and more.
• Mlflow Deployment Controller - Listens MLFlow model registry changes and deploy models based on configurations.
• Truss - Serve any model without boilerplate code. (HN) (Docs)
• Remote Runner - Easy pythonic way to migrate your python training scripts from a local environment to a powerful cloud-backed instance.
• BMList - List of big pre-trained models (GPT-3, DALL-E2...).
• ModelBox - Extensible machine learning model store and model transformation and distribution service.
• Run Stable Diffusion on Your M1 Mac’s GPU (2022) (HN)
• Stable Diffusion Dream Script - Fork of CompVis/stable-diffusion, the wonderful open source text-to-image generator.
• Exporters - Export Hugging Face models to Core ML and TensorFlow Lite.
• SD-explorer - Toy project to explore Stable Diffusion locally through a nodeJS server.
• MultiNeRF: A Code Release for Mip-NeRF 360, Ref-NeRF, and RawNeRF
• Prog Rock Stable - Stable Diffusion with some Proggy Enhancements.
• Diffusers-Interpret - Model explainability tool built on top of Diffusers.
• Using GitHub as Artifactory for Machine Learning Model Artifacts (2022)
• Replicate - Run open-source machine learning models with a cloud API. (Go Replicate) (GitHub)
• dfserver - Distributed backend AI pipeline server.
• dstack - Command-line utility to provision infrastructure for ML workflows. (Docs)
• Privacy Meter - Open-source library to audit data privacy in statistical and machine learning algorithms.
• fastDeploy - Deploy DL/ ML inference pipelines with minimal extra code.
• Modelverse - Content-Based Search for Deep Generative Models. (Code)
• FlagAI - Fast, easy-to-use and extensible toolkit for large-scale model.
• Stochastic - AI Acceleration Platform. (GitHub)
• voltaML - Lightweight library to convert and run your ML/DL deep learning models in high performance inference runtimes like TensorRT, TorchScript, ONNX and TVM.
• AWS Deep Learning Containers - Docker images for training and serving models in TensorFlow, TensorFlow 2, PyTorch, and MXNet.
• Spotty - Training deep learning models on AWS and GCP instances.
• FastDeploy - Accessible and efficient deployment Development Toolkit.
• explainerdashboard - Quickly build Explainable AI dashboards that show the inner workings of so-called "blackbox" machine learning models.
• MatxScript - Model pre- and post-processing framework.
• Zoltar - Common library for serving TensorFlow, XGBoost and scikit-learn models in production.
• Robust Intelligence - Machine learning models fail. Prevent bad outcomes with the only end-to-end solution.
• Banana Serverless - Framework to serve ML models in production using simple HTTP servers.
• SkyPilot - Framework for easily running machine learning workloads on any cloud through a unified interface.
• m2cgen (Model 2 Code Generator) - Lightweight library which provides an easy way to transpile trained statistical models into a native code.
• Models - Hugging Face
• Training One Million Machine Learning Models in Record Time with Ray (2022) (Tweet)
• Do Simpler Machine Learning Models Exist and How Can We Find Them? (2022) (HN)
• Train and deploy a DreamBooth model on Replicate (2022) (Code)
• Reference models and tools for Cloud TPUs
• Energon AI - Large-scale model inference.
• Web AI - Run modern deep learning models in the browser.
• Web Transformers - Transformer neural networks in the browser.
• How to Get 1.5 TFlops of FP32 Performance on a Single M1 CPU Core (HN)
• ClearML - Model-Serving Orchestration and Repository Solution. (Web)
• Cloud Pipelines Editor - Web app that allows the users to build and run Machine Learning pipelines without having to set up development environment.
• Slapo - Schedule language for progressive optimization of large deep learning model training.
• How DoorDash Upgraded a Heuristic with ML to Save Thousands of Canceled Orders (2023)
• featureimpact - Python package for estimating the impact of features on ML models.
• Deep Learning Tuning Playbook - Playbook for systematically maximizing the performance of deep learning models.
• UpTrain - Open-source ML observability and refinement tool. (HN)
• FastQL Inference Server - Spin up a blazing fast Rust GraphQL API to serve your ML model in one line of Python code.
• Zeno - Interactive framework for machine learning evaluation.
• Coadaptive Harness for Effective Evaluation, Steering, & Enhancement
• How to train large models on many GPUs? (2021) (HN)
• ML Model parallelism 101 - Functional local implementations of main model parallelism approaches.
• Minimal ML Template - Minimal ml project template that uses HF transformers and wandb to train a simple NN and evaluate it, in a stateless manner compatible with Spot instances Kubernetes workflows.
• BMTrain - Efficient Training (including pre-training and fine-tuning) for Big Models.
• CIFAR10 hyperlightspeedbench - Train to 94% on CIFAR-10 in less than 7 seconds on a single A100, the current world record.
• Transformers.js - Run Transformers in your browser. (HN)
• ailia SDK - Cross-platform high speed inference SDK.
• Large Audio Models - Keep track of big models in audio domain, including speech, singing, music etc.
• BlindAI - Fast, easy-to-use, and confidential inference server, allowing you to easily and quickly deploy your AI models.
• Cformers - SoTA Transformers with C-backend for fast inference on your CPU.
• Triton Tutorials - Tutorials and examples for Triton Inference Server.
• Effortlessly Build Machine Learning Apps with Hugging Face’s Docker Spaces (2023)
• Navi - High-Performance Machine Learning Serving Server in Rust. (Reddit)
• Dreamlook.ai - Train models in minutes. Scale up to 1000s of runs per day.
• Flow Coder - Compile ML models into dependency-free source code for easy deployment.
• Window - Use your own AI models on the web.
• Motion - Framework for building ML pipelines in Python.
• CI/CD for Machine Learning Models
• ONNX Simplifier - Simplify your onnx model.
• Temporal quality degradation in AI models (2023) (HN)
• AzureML and Azure OpenAI - Sample project that demonstrates how to use Azure Machine Learning to fine-tune and deploy a model using Azure OpenAI.
• Banana Node/TypeScript SDK
• Meerkat - Interactive data structures for evaluating foundation models.
• AutoGluon-Cloud - User tools to train, fine-tune and deploy AutoGluon backed models on the cloud.
• Hidet: A Deep Learning Compiler for Efficient Model Serving (2023) (HN)
• MLC LLM - Universal solution that allows any language models to be deployed natively on a diverse set of hardware backends and native applications. (HN)
• Ask HN: What do you use for ML Hosting? (2023)
• SageMaker Inference Toolkit - Serve machine learning models within a Docker container using Amazon SageMaker.
• MTIA v1: Meta's first-generation AI inference accelerator (2023)
• C Transformers - Python bindings for the Transformer models implemented in C/C++ using GGML library.
• Full example of GPTQ 4-bit inference on Modal
• Olive - Easy-to-use hardware-aware model optimization tool that composes industry-leading techniques across model compression, optimization, and compilation.
• It’s infuriatingly hard to understand how closed models train on their input (2023) (HN)
• Autodistill - Images to inference with no labeling (use foundation models to train supervised models).
• FastTrackML - Rewrite of the MLFlow tracking server with a focus on scalability.
• Transformers.js on Netlify
• Self-hostable ML manager
• LLM Tracker
• Model Transparency - Utilities and examples related to the security of machine learning pipelines.