MLOps for Developers: Deploying, Monitoring, and Optimizing Machine Learning Models

The Model That Worked in Jupyter and Failed in Production

In 2020, a team at a financial services company trained a fraud detection model that achieved 99.2% accuracy in their Jupyter notebooks. They deployed it to production. Within a week, false positives had increased by 300%, and the company was blocking legitimate transactions from thousands of customers. The model wasn't broken — the data had changed. The pandemic had shifted spending patterns so dramatically that the model's training data no longer resembled reality[google-mlops].

This is the central lesson of MLOps: a model is not a feature you ship once. It's a system that degrades over time. Unlike traditional software, where a bug either exists or doesn't, ML models silently degrade as the world changes around them. The data drifts. The distribution shifts. The model that was accurate last month becomes inaccurate this month — and without monitoring, you won't know until customers complain[mlops-org].

MLOps (Machine Learning Operations) is the set of practices that keeps ML models working in production: deploying them reliably, monitoring them continuously, retraining them when they degrade, and doing all of this at scale. This guide covers the fundamentals: choosing the right hardware for inference, optimizing costs (cloud API vs self-hosting), detecting model and data drift, and building CI/CD pipelines that work for ML systems.

The ML Lifecycle: From Experiment to Production

The path from a Jupyter notebook to a production ML system has more stages than most developers expect[google-mlops]:

1. Data collection & preparation — gathering, cleaning, labeling, and splitting data
2. Experimentation — training models, tuning hyperparameters, evaluating on validation sets
3. Model validation — testing on holdout sets, checking for bias, verifying performance metrics
4. Deployment — packaging the model, setting up inference infrastructure, exposing an API
5. Monitoring — tracking prediction latency, error rates, data drift, and model drift
6. Retraining — collecting new data, retraining the model, validating, redeploying

Traditional software has steps 1-4 and mostly skips 5-6. ML systems need all six — and steps 5-6 are where most teams fail. A model deployed without monitoring is a model that will fail silently[mlops-org].

The key insight: the experiment (steps 1-3) is often less than 20% of the total effort. The production infrastructure (steps 4-6) is where the real engineering happens[google-mlops].

Infrastructure: GPU Selection and VRAM Requirements

Deploying an ML model requires hardware — and for most modern models, that means GPUs. The central question: how much VRAM do I need?

VRAM (Video RAM) is the memory on a GPU where the model weights and activations live during inference. A 7B-parameter model in 16-bit precision requires roughly 14 GB just for the weights (7 billion parameters × 2 bytes each). Add activations, KV cache, and overhead, and a 7B model needs roughly 16-20 GB of VRAM for comfortable inference. A 70B model needs roughly 140-160 GB — which means multiple high-end GPUs[huggingface-docs].

The GPU landscape (2026):

Quantization reduces the precision of model weights to fit more into less VRAM. 8-bit quantization halves the memory requirement (7B model → ~7 GB). 4-bit quantization quarters it (7B model → ~3.5 GB). The cost: a small accuracy degradation that's often imperceptible for inference tasks[huggingface-docs].

The LLM Hardware Requirements Calculator estimates VRAM needs based on model size, precision (16-bit, 8-bit, 4-bit), and sequence length. Use it before buying hardware — the difference between a $300 RTX 5060 and a $15,000 H100 is whether your model fits in VRAM at all.

Cost Optimization: Cloud API vs Self-Hosting

The most consequential infrastructure decision for ML deployment is: should I call a cloud API or run the model myself?

Cloud APIs (OpenAI, Anthropic, Google) charge per token. Current flagship pricing (July 2026):

For an application processing 10 million tokens per day using GPT-5.4 (mix of input/output), that's roughly $25-75 per day, or $750-2,250 per month. The benefit: zero infrastructure management, automatic scaling, and access to the latest models. The cost scales linearly with usage — there's no way to amortize hardware[paperswithcode].

Self-hosting requires buying or renting GPUs but eliminates per-token costs. A $10,000 A100 amortized over 3 years costs roughly $275/month. Electricity and hosting add $100-200/month. Total: ~$400-500/month for unlimited inference. The break-even point depends on your token volume and the specific model — roughly 1-3 million tokens per day for current frontier models[mlops-org].

The LLM API Cost Calculator and Local LLM Break-Even Calculator compute the monthly cost for both approaches given your expected token volume. Use them to make the business case before committing to an architecture.

The hybrid approach (common in production): use cloud APIs for development, testing, and low-volume tasks; self-host for high-volume, latency-sensitive, or privacy-critical workloads. This gives you the flexibility of cloud APIs with the cost efficiency of self-hosting where it matters[google-mlops].

Monitoring: Detecting Silent Degradation

A model in production is like a sensor that slowly goes out of calibration. The inputs change (data drift), the relationship between inputs and outputs changes (concept drift), and the model's predictions become less accurate over time. Without monitoring, you won't notice until the business impact is severe[mlops-org].

CI/CD for ML: Pipelines That Handle Models

Traditional CI/CD (Continuous Integration/Continuous Deployment) automates building, testing, and deploying code. ML systems need additional steps because the "code" includes data, model weights, and hyperparameters — all of which can change independently[google-mlops].

A typical ML pipeline:

1. Data validation — check that new training data matches expected schema, distributions, and quality
2. Data preprocessing — clean, normalize, and split data
3. Model training — train the model with the current hyperparameters
4. Model evaluation — compare the new model's performance to the current production model
5. Model validation — check for bias, fairness, and edge cases
6. Model packaging — export to a deployable format (ONNX, TorchScript, TensorFlow SavedModel)
7. Deployment — roll out to production (canary, blue-green, or shadow deployment)
8. Monitoring — track the new model's performance in production

The key difference from traditional CI/CD: the model evaluation step compares the new model to the current model. If the new model doesn't improve performance (or degrades it), the pipeline should reject the deployment — even if all unit tests pass[google-mlops].

Deployment strategies:

• Canary deployment: route 5% of traffic to the new model, monitor for errors, then gradually increase
• Blue-green deployment: run two identical environments, switch traffic atomically
• Shadow deployment: run the new model in parallel with the old, comparing outputs without affecting users

Rollbacks are critical. If the new model degrades in production, you need to revert to the previous version immediately. This requires keeping the previous model version available and having a fast rollback mechanism[mlops-org].

Practical Tips for MLOps

1. Version everything. Data, code, model weights, hyperparameters, and environment configurations should all be versioned. A model is not reproducible if you can't reconstruct the exact training environment.

2. Monitor from day one. Deploying a model without monitoring is shipping a bug you can't detect. Set up latency, error rate, and distribution monitoring before the model goes live.

3. Use feature stores. A feature store centralizes the features used for training and inference, ensuring consistency between the two. Without it, training uses one set of features and production uses another — a subtle but devastating bug.

4. Automate retraining. Manual retraining is retraining that doesn't happen. Set up pipelines that retrain on schedule and trigger on drift detection.

5. Test for edge cases. ML models fail on inputs that are rare in training data but common in production (unusual image formats, empty strings, extreme values). Test explicitly for these.

6. Plan for cold starts. A new deployment has no cache, no JIT compilation, no warm connections. The first requests will be slow. Pre-warm the model before routing production traffic to it.

7. Document model cards. A model card describes what the model does, what data it was trained on, its known limitations, and its performance characteristics. Future you (and your team) will thank you.

Limitations: MLOps Is Not a Silver Bullet

MLOps practices keep models working, but they don't solve fundamental problems:

• Bad data → bad model. No amount of MLOps can compensate for biased, incomplete, or unrepresentative training data.
• Wrong problem → wrong model. If you're predicting the wrong thing, a well-deployed model is a well-deployed mistake.
• Regulatory compliance. MLOps doesn't automatically make your model GDPR-compliant or FDA-approved. Compliance requires additional processes.
• Explainability. A model that works but can't be explained may be unacceptable in regulated industries (healthcare, finance, criminal justice).

MLOps is necessary but not sufficient. It keeps good models working — it doesn't make bad models good.

Frequently Asked Questions

❓ What is MLOps?

✅ MLOps (Machine Learning Operations) is the set of practices for deploying, monitoring, and maintaining ML models in production. It extends traditional DevOps to handle the unique challenges of ML: data drift, model degradation, and experiment tracking.

❓ How much VRAM do I need for a 7B model?

✅ A 7B-parameter model in 16-bit precision requires roughly 14 GB for weights alone, plus 2-6 GB for activations and KV cache. Total: 16-20 GB. With 8-bit quantization, you can fit it in 8-10 GB. With 4-bit, 4-6 GB.

❓ Should I use a cloud API or self-host?

✅ Cloud APIs (OpenAI, Anthropic) are cheaper for low-volume usage and eliminate infrastructure management. Self-hosting is cheaper at high volumes (typically >1-3 million tokens/day). The break-even depends on your token volume and the specific model.

❓ What is data drift?

✅ Data drift occurs when the distribution of input data changes over time. A model trained on last year's data may not work well on this year's data. Detection methods include PSI (Population Stability Index) and statistical tests like Kolmogorov-Smirnov.

❓ What is model drift?

✅ Model drift (concept drift) occurs when the relationship between inputs and outputs changes. Even if input data looks the same, the model's predictions become less accurate. Detection requires monitoring prediction distributions and comparing to ground truth when available.

❓ How often should I retrain?

✅ Retrain on a schedule (weekly or monthly) and additionally trigger retraining when drift exceeds a threshold. The right frequency depends on how fast your data changes — fraud models may need daily retraining; content recommendation models may need weekly.

❓ What is quantization?

✅ Quantization reduces the precision of model weights to fit more into less VRAM. 8-bit halves the memory requirement; 4-bit quarters it. The cost is a small accuracy degradation that's often imperceptible for inference. Use it to run larger models on consumer hardware.

❓ What is a canary deployment?

✅ A deployment strategy where you route a small percentage of traffic (e.g., 5%) to the new model, monitor for errors, then gradually increase. If something goes wrong, only a small fraction of users are affected.