MLOps Workshop Demo

End-to-end Azure ML + MLflow hands-on workshop demonstrating production MLOps patterns.

🎯 What You'll Learn

Module	Topic	Key Concepts	Azure Services	Difficulty
1	Environment Setup	SDK auth, workspace config	Azure ML Workspace, Key Vault	Beginner
2	Train & Track with MLflow	Experiment runs, metrics, artifacts	MLflow Tracking, Model Registry	Beginner
3	Batch Endpoint Deployment	Model deployment, scoring	Batch Endpoints, Compute Clusters	Intermediate
4	Automated Retraining Pipeline	Pipeline components, orchestration	Azure ML Pipelines	Intermediate
5	Drift Detection (Observability)	PSI, JSD, data quality metrics	Jobs, Metrics Logging	Intermediate
6	Governance & Audit Logging	Activity logs, compliance	Azure Monitor, RBAC	Beginner

📁 Project Structure

├── notebooks/
│   └── 01_azure_ml_mlflow_quickstart.ipynb   # Main workshop notebook
├── 01-automated-retraining/                   # Retraining pipeline
│   ├── pipeline.py                            # Azure ML pipeline definition
│   ├── submit_pipeline.py                     # Submit pipeline job
│   └── components/                            # Pipeline steps
├── 02-observability/                          # Drift detection
│   ├── drift_report.py                        # PSI/JSD drift metrics
│   └── submit_drift_job.py                    # Submit drift job
├── 03-governance/                             # Audit & compliance
│   ├── audit_logging.py                       # Azure Activity Log queries
│   └── run_audit_report.py                    # Generate audit report
├── 04-feature-store/                          # Feature store assets
├── infra/                                     # Bicep templates
├── MLOps_Workshop_Playbook.html               # Interactive workshop guide
└── requirements.txt                           # Python dependencies

🚀 Quick Start

Prerequisites

• Azure subscription with Contributor access
• Python 3.9+
• VS Code with Python & Jupyter extensions
• Azure CLI (az login)

1. Setup Environment

# Clone repo
git clone https://github.com/ritwickmicrosoft/mlops-workshop-demo.git
cd mlops-workshop-demo

# Create virtual environment
python -m venv .venv311
.venv311\Scripts\Activate.ps1
pip install -r requirements.txt

2. Configure Azure

az login
$env:AZURE_SUBSCRIPTION_ID = "<your-subscription-id>"
$env:AZURE_RESOURCE_GROUP = "rg-dnd-mlops-demo"
$env:AZURE_ML_WORKSPACE = "mlw-dndmlops2-dev"

3. Run the Workshop

Option A: Interactive Playbook (recommended)

• Open the Workshop Playbook in your browser
• Follow the step-by-step guide with checkboxes

Option B: Notebook Only

• Open notebooks/01_azure_ml_mlflow_quickstart.ipynb
• Run cells sequentially

⚠️ Known Issues & Fixes

NumPy 2.x Compatibility

MLflow models logged with NumPy 2.x fail on Azure ML endpoints (which use NumPy 1.x).

Fix: Re-log the model with explicit pip requirements:

mlflow.sklearn.log_model(
    model,
    artifact_path='model',
    pip_requirements=['numpy<2.0', 'scikit-learn>=1.0,<2.0', 'mlflow'],
)

See notebook cells 11-12 for the complete fix.

📊 Key Metrics

Drift Detection (Module 5)

Metric	Low Risk	Medium	High Risk
PSI (Population Stability Index)	< 0.1	0.1-0.25	> 0.25
JSD (Jensen-Shannon Divergence)	< 0.05	0.05-0.1	> 0.1

🔧 Azure Resources

Resource	Purpose
Azure ML Workspace	Experiments, models, endpoints
Compute Cluster (cpu-cluster)	Training & batch jobs
Model Registry	Versioned model storage
Batch Endpoint	Asynchronous inference

📚 References

• Azure ML Documentation
• MLflow with Azure ML
• Batch Endpoints

🧹 Cleanup

# Delete batch endpoint
az ml batch-endpoint delete -n spam-batch-* -g $env:AZURE_RESOURCE_GROUP -w $env:AZURE_ML_WORKSPACE --yes

# Delete online endpoint (if created)
az ml online-endpoint delete -n spam-clf-* -g $env:AZURE_RESOURCE_GROUP -w $env:AZURE_ML_WORKSPACE --yes

---

Workshop Playbook: Open Interactive Guide