Machine Learning Ops (MLOps) and Data Science

Architectural Context

Detailed reference for Machine Learning Ops (MLOps) and Data Science in the context of AI.

Standard Reference

• cloudblogs.microsoft.com: Simple steps to create scalable processes to deploy ML models as microservices [COMMUNITY-TOOL]
• rubrix ⭐ 4981 [ENTERPRISE-STABLE]
• infoworld.com: 13 open source projects transforming AI and machine learning [COMMUNITY-TOOL]
• semaphoreci.com: Why Do We Need DevOps for ML Data? [COMMUNITY-TOOL]
• mlops.community: MLOps with Flyte: The Convergence of Workflows Between Machine Learning and Engineering [COMMUNITY-TOOL]
• mlops.community: MLOps Simplified: orchestrating ML pipelines with infrastructure abstraction. Enabled by Flyte [COMMUNITY-TOOL]
• docs.microsoft.com: MLflow and Azure Machine Learning [COMMUNITY-TOOL]
• canvatechblog.com: Supporting GPU-accelerated Machine Learning with Kubernetes and Nix [COMMUNITY-TOOL]
• Nix [COMMUNITY-TOOL] — - github.com/NVIDIA/nvidia-docker: NVIDIA/nvidia-docker/volumes.go NVIDIA’s documentation is disappointingly evasive on what the “driver” is, but we find a good answer in their official source code.
• github.com/meta-llama/llama-recipes ⭐ 18334 [DE FACTO STANDARD]
• docs.microsoft.com: Machine Learning Experimentation in VS Code with DVC Extension [COMMUNITY-TOOL]
• github.com/CASIA-IVA-Lab/FastSAM ⭐ 8342 [ENTERPRISE-STABLE]
• github.com/VikParuchuri/surya ⭐ 19766 [DE FACTO STANDARD]
• github.com/decodingml: Real-time news search engine using Upstash Kafka and Vector DB ⭐ 139 [COMMUNITY-TOOL]
• github: A very Long never ending Learning around Data Engineering & Machine' Learning [COMMUNITY-TOOL]
• cd.foundation: Announcing the CD Foundation MLOps SIG [COMMUNITY-TOOL]
• dafriedman97.github.io: Machine Learning from Scratch [COMMUNITY-TOOL]
• cortex.dev: How to build a pipeline to retrain and deploy models [COMMUNITY-TOOL]
• towardsdatascience.com: A Kubernetes architecture for machine learning web-application' deployments [COMMUNITY-TOOL]
• cloud.google.com: How to use a machine learning model from a Google Sheet' using BigQuery ML [COMMUNITY-TOOL]
• itnext.io: Building ML Componentes on Kubernetes [COMMUNITY-TOOL]
• towardsdatascience.com: Deploying An ML Model With FastAPI — A Succinct' Guide [COMMUNITY-TOOL]
• ML Platform Workshop ⭐ 445 [COMMUNITY-TOOL]
• towardsdatascience.com: Automatically Generate Machine Learning Code with' Just a Few Clicks [COMMUNITY-TOOL]
• towardsdatascience.com: Schemafull streaming data processing in ML pipelines [COMMUNITY-TOOL]
• analyticsindiamag.com: Top tools for enabling CI/CD in ML pipelines [COMMUNITY-TOOL]
• towardsdatascience.com: Step-by-step Approach to Build Your Machine Learning' API Using Fast API [COMMUNITY-TOOL]
• ravirajag.dev: MLOps Basics - Week 10: Summary [COMMUNITY-TOOL]
• medium.com/workday-engineering: Implementing a Fully Automated Sharding' Strategy on Kubernetes for Multi-tenanted Machine Learning Applications [COMMUNITY-TOOL]
• medium.com/globant: Advantages of Deploying Machine Learning models with' Kubernetes 🌟 [COMMUNITY-TOOL]
• medium.com/pythoneers: MLOps: Tool Stack Requirement in Machine Learning' Pipeline [COMMUNITY-TOOL]
• medium.com/formaloo: How no-code platforms are democratizing data science' and software development 🌟 [COMMUNITY-TOOL]
• towardsdatascience.com: From Jupyter Notebooks to Real-life: MLOps 🌟 [COMMUNITY-TOOL]
• datarevenue.com: Airflow vs. Luigi vs. Argo vs. MLFlow vs. KubeFlow [COMMUNITY-TOOL]
• towardsdatascience.com: From Dev to Deployment: An End to End Sentiment' Classifier App with MLflow, SageMaker, and Streamlit [COMMUNITY-TOOL]
• elconfidencial.com: La batalla entre Google y Meta que nadie esperaba: revolucionar' la biología 🌟 [COMMUNITY-TOOL]
• swirlai.substack.com: SAI #08: Request-Response Model Deployment - The MLOps' Way, Spark - Executor Memory Structure and more... 🌟 [COMMUNITY-TOOL]
• youtube: Making Friends with Machine Learning | Cassie Kozyrkov | playlist' 🌟 [COMMUNITY-TOOL]
• openai.com: Scaling Kubernetes to 7,500 nodes 🌟 [COMMUNITY-TOOL]
• huyenchip.com: Building LLM applications for production [COMMUNITY-TOOL]
• medium.com/@study.uttam: Main Challenges of Machine Learning [COMMUNITY-TOOL]
• learn.microsoft.com: Machine Learning operations maturity model 🌟 [COMMUNITY-TOOL]
• medium.com/ai-hero: Streamlining Machine Learning Operations (MLOps) with' Kubernetes and Terraform [COMMUNITY-TOOL]
• medium.com/@karanshingde: Machine Learning in Production— Your Comprehensive' 101 Practical Guide [COMMUNITY-TOOL]
• marvelousmlops.substack.com: CI/CD for MLOps on GitLab (part 1) [COMMUNITY-TOOL]
• medium.com/aiguys: MLOps: Serving AI apps to million users [COMMUNITY-TOOL]
• marvelousmlops.substack.com: How to sell MLOps in large Organizations [COMMUNITY-TOOL]
• marvelousmlops.substack.com: MLOps roadmap 2024 [COMMUNITY-TOOL]
• towardsdatascience.com: Deploying LLM Apps to AWS, the Open-Source Self-Service' Way [COMMUNITY-TOOL]
• axelmendoza.com: The Ultimate Guide To ML Model Deployment In 2024 [COMMUNITY-TOOL]
• towardsdatascience.com: Build Machine Learning Pipelines with Airflow and' Mlflow: Reservation Cancellation Forecasting [COMMUNITY-TOOL]
• marvelousmlops.substack.com: Technical roles in Data Science: Who is doing' what? [COMMUNITY-TOOL]
• marvelousmlops.substack.com: Traceability & Reproducibility [COMMUNITY-TOOL]
• marvelousmlops.substack.com: Learn Machine Learning and Neural Networks' without Frameworks [COMMUNITY-TOOL]
• seattledataguy.substack.com: Data Engineering Vs Machine Learning Pipelines [COMMUNITY-TOOL]
• aiml.com: Large Language Models Quiz (Medium) [COMMUNITY-TOOL]
• medium.com/@samiullah6799: Different Roles in MLOps [COMMUNITY-TOOL]
• dev.to/pavanbelagatti: Deploy Any AI/ML Application On Kubernetes: A Step-by-Step' Guide! [COMMUNITY-TOOL]
• marvelousmlops.substack.com: Sharpen your cookiecutter: speed up repo creation' with workflows [COMMUNITY-TOOL]
• decodingml.substack.com: How to ensure your models are fail-safe in production? [COMMUNITY-TOOL]
• freecodecamp.org: MLOps Course – Learn to Build Machine Learning Production' Grade Projects [COMMUNITY-TOOL]
• medium.com/@kevin30101999: Machine Learning Pipeline using Argo workflow' 🌟 [COMMUNITY-TOOL]
• roadmap.sh: MLOps roadmap [COMMUNITY-TOOL]
• Marvelous MLOps Substack [COMMUNITY-TOOL]
• decodingml.substack.com: Decoding ML Newsletter [COMMUNITY-TOOL]
• youtube.com: Optimizing LLM Training with Airbnb's Next-Gen ML Platform [COMMUNITY-TOOL]
• Ray [COMMUNITY-TOOL]
• medium.com/mlearning-ai: The Best Object Detection Libraries That I Work' With [COMMUNITY-TOOL]
• artifacthub.io: mlflow-server [COMMUNITY-TOOL]
• pypi.org/project/airflow-provider-mlflow [COMMUNITY-TOOL]
• Kubeflow [COMMUNITY-TOOL]
• infracloud.io: Machine Learning Orchestration on Kubernetes using Kubeflow [COMMUNITY-TOOL]
• blog.devgenius.io: Kubeflow Cloud Deployment (AWS) [COMMUNITY-TOOL]
• joseprsm.medium.com: How to build Machine Learning models that train themselves [COMMUNITY-TOOL]
• medium.com/dkatalis: Creating a Mutating Webhook for Great Good! Or: how' to automatically provision Pods on a specific node pool [COMMUNITY-TOOL]
• Union Cloud [COMMUNITY-TOOL]
• Machine Learning in Production. What does an end-to-end ML workflow look like in production? (transcript) 🌟🌟🌟 [COMMUNITY-TOOL]
• stackoverflow.com: How is Flyte tailored to "Data and Machine Learning"? [COMMUNITY-TOOL]
• union.ai: Production-Grade ML Pipelines: Flyte™ vs. Kubeflow [COMMUNITY-TOOL]
• medium.com/@timleonardDS: Who Let the DAGs out? Register an External DAG' with Flyte (Chapter 3) [COMMUNITY-TOOL]
• aws.amazon.com: MLOps foundation roadmap for enterprises with Amazon SageMaker [COMMUNITY-TOOL]
• aws.amazon.com: Promote pipelines in a multi-environment setup using Amazon' SageMaker Model Registry, HashiCorp Terraform, GitHub, and Jenkins CI/CD [COMMUNITY-TOOL]
• bea.stollnitz.com: Creating batch endpoints in Azure ML [COMMUNITY-TOOL]
• blog.devops.dev: Mastering Machine Learning at Scale with Azure Machine' Learning [COMMUNITY-TOOL]
• youtube: Deploy Convolutional Neural Network (CNN) on Azure with Python' | Deep Learning Deployment | MLOPS [COMMUNITY-TOOL]
• learn.microsoft.com: Azure Well-Architected Framework perspective on Azure' Machine Learning [COMMUNITY-TOOL]
• marvelousmlops.substack.com: Model serving architectures on Databricks [COMMUNITY-TOOL]
• medium.com/sync-computing: Top 9 Lessons Learned about Databricks Jobs Serverless [COMMUNITY-TOOL]
• thenewstack.io: KServe: A Robust and Extensible Cloud Native Model Server [COMMUNITY-TOOL]
• medium.com/bakdata: Scalable Machine Learning with Kafka Streams and KServe [COMMUNITY-TOOL]
• analyticsvidhya.com: Bring DevOps To Data Science With MLOps [COMMUNITY-TOOL]
• analyticsindiamag.com: Is coding necessary to work as a data scientist? [COMMUNITY-TOOL]
• redhat.com: Introducing Red Hat OpenShift Data Science [COMMUNITY-TOOL]
• towardsdatascience.com: From DevOps to MLOPS: Integrate Machine Learning' Models using Jenkins and Docker [COMMUNITY-TOOL]
• catalog.ngc.nvidia.com: NVIDIA GPU Operator - Helm chart 🌟🌟🌟 [COMMUNITY-TOOL]
• jimangel.io: A Practical Guide to Running NVIDIA GPUs on Kubernetes [COMMUNITY-TOOL]
• huggingface.co: Implementing Fractional GPUs in Kubernetes with Aliyun Scheduler [COMMUNITY-TOOL]
• medium.com/@bchenjh: Distributed full fine-tuning of Llama2 on Kubernetes [COMMUNITY-TOOL]
• bodywork-ml/bodywork-core: Bodywork ⭐ 436 [COMMUNITY-TOOL]
• learn.iterative.ai: Iterative Tools for Data Scientists & Analysts [COMMUNITY-TOOL]
• DVC [COMMUNITY-TOOL]
• tensorchord/envd: Reproducible development environment for AI/ML 🌟 ⭐ 2206 [COMMUNITY-TOOL]
• postgresml/postgresml 🌟 ⭐ 6791 [ENTERPRISE-STABLE]
• blog.devgenius.io: Training model with Jenkins using docker: MLOPS [COMMUNITY-TOOL]
• vaex.io [COMMUNITY-TOOL]
• thenewstack.io: 7 Must-Have Python Tools for ML Devs and Data Scientists' 🌟 [COMMUNITY-TOOL]
• github.com/SymbioticLab/Oobleck: Oobleck - Resilient Distributed Training' Framework ⭐ 100 [COMMUNITY-TOOL]
• github.com/aimhubio/aim ⭐ 6126 [ENTERPRISE-STABLE]
• github.com/XuehaiPan/nvitop 🌟 ⭐ 6921 [ENTERPRISE-STABLE]
• github.com/Netflix/metaflow 🌟 ⭐ 10107 [ENTERPRISE-STABLE]
• zenml.io: ZenML [COMMUNITY-TOOL]
• betterprogramming.pub: Attach a Visual Debugger to ML-training Jobs on Kubernetes [COMMUNITY-TOOL]
• fepegar/vesseg ⭐ 44 [COMMUNITY-TOOL]
• github.com/10tanmay100: MEDICAL-DATA-PROJECT-END2END-WITH-FEW-MLOPS ⭐ 3 [COMMUNITY-TOOL]
• dair-ai/ML-Course-Notes: ML Course Notes 🌟 ⭐ 6455 [ENTERPRISE-STABLE]
• Kaggle Competitions [COMMUNITY-TOOL]
• kaggle.com: Sports Car Prices dataset [COMMUNITY-TOOL]
• isic-archive.com [COMMUNITY-TOOL]
• freecodecamp.org: How to Download a Kaggle Dataset Directly to a Google' Colab Notebook [COMMUNITY-TOOL]

---

💡 Explore Related: AI | ChatGPT | AI Agents MCP