Imagine your AI model is a student who predicted exam scores. The real score was 80, but the model said 75. It was off by 5 marks. Now imagine the model predicted 500 students' scores. How do you measure how wrong it was — overall?
That is exactly what ME, MAE, and MAPE do. They are three simple ruler-like tools that tell you how far off your model's predictions are from the truth.
💡 Why do MLOps engineers care about these? In production, you must constantly measure how good your model is on live data. ME, MAE, and MAPE are the first metrics you check every single morning in any serious ML monitoring dashboard. They are the pulse rate of your deployed model. 💓
Part 1: The Big Idea — What is a Prediction Error? 🎯
Before we learn ME, MAE, and MAPE, we need to understand one simple concept: Error.
Error is simply the difference between what your model predicted and what the actual answer really was.
Error = Actual Value − Predicted Value
Example:
─────────────────────────────────────────────────────────
Actual temperature tomorrow: 28°C
Model predicted: 25°C
Error: 28 − 25 = +3°C (under-predicted)
Actual temperature tomorrow: 28°C
Model predicted: 31°C
Error: 28 − 31 = −3°C (over-predicted)
─────────────────────────────────────────────────────────
Positive error → model predicted TOO LOW (under-estimate)
Negative error → model predicted TOO HIGH (over-estimate)
Zero error → model was PERFECT ✅
Now, we never just have one prediction — we have hundreds or thousands. So we need a way to summarise all those individual errors into one number. That is exactly what ME, MAE, and MAPE do — each in a slightly different way!
Part 2: ME — Mean Error 🔵
What is ME? (The 10-Year-Old Explanation)
Imagine an archer shooting 10 arrows at a target. Some arrows land to the left of the bullseye (negative error). Some land to the right (positive error).
Mean Error is like asking: "On average, is the archer leaning left or right?" It does NOT care about how far each arrow missed — it only cares about the direction of the overall miss.
ME Formula:
ME = (1/n) × Σ (Actual_i − Predicted_i)
Where:
n = total number of predictions
Σ = sum of all errors
Actual_i = the real true value for prediction i
Predicted_i = what the model predicted for i
In plain English:
ME = Average of all (Actual − Predicted) values
─────────────────────────────────────────────────────────────
Day Actual Predicted Error (Actual − Predicted)
─────────────────────────────────────────────────────────────
Mon 100 95 +5 (model under-predicted)
Tue 200 210 −10 (model over-predicted)
Wed 150 148 +2
Thu 300 305 −5
Fri 250 252 −2
─────────────────────────────────────────────────────────────
Sum of errors = 5 + (−10) + 2 + (−5) + (−2) = −10
ME = −10 / 5 = −2.0
─────────────────────────────────────────────────────────────
ME = −2.0 means: on average, the model over-predicts by 2 units.
This is called BIAS.
The Danger of ME — Errors Cancel Each Other Out!
Here is the tricky part about ME. If some errors are positive (+5) and others are negative (−5), they cancel each other out and ME becomes zero — even though the model was wrong every single time!
Misleading ME Example:
Day Actual Predicted Error
Mon 100 110 −10
Tue 100 90 +10
Wed 100 115 −15
Thu 100 85 +15
ME = (−10 + 10 − 15 + 15) / 4 = 0 / 4 = 0.0
ME says: "Perfect model!" 🎉
Reality: The model was WRONG every single day! 😱
When IS Mean Error Useful?
ME is most useful for detecting systematic bias in your model.
- A consistently negative ME means your model always over-predicts — it is biased high. In a pricing model, this could mean customers always see inflated prices. 💸
- A consistently positive ME means your model always under-predicts — it is biased low. In a demand forecasting model, you would always order too little stock. 📦
- An ME close to zero means the model has no systematic bias — errors are roughly balanced. Good sign! ✅
Part 3: MAE — Mean Absolute Error 🟠
What is MAE? (The 10-Year-Old Explanation)
Remember the archer problem? ME told us the direction the archer leans. But what if we just want to know: "How far did each arrow miss — no matter which direction?"
That is Mean Absolute Error. The word Absolute means we ignore the + or − sign and only care about the size of the mistake. A miss of −10 becomes just 10. A miss of +7 stays 7.
MAE Formula:
MAE = (1/n) × Σ |Actual_i − Predicted_i|
The | | bars mean "take the absolute value" — always make it positive!
─────────────────────────────────────────────────────────────
Day Actual Predicted Error |Error| (Absolute)
─────────────────────────────────────────────────────────────
Mon 100 95 +5 5
Tue 200 210 −10 10
Wed 150 148 +2 2
Thu 300 305 −5 5
Fri 250 252 −2 2
─────────────────────────────────────────────────────────────
Sum of |Errors| = 5 + 10 + 2 + 5 + 2 = 24
MAE = 24 / 5 = 4.8
─────────────────────────────────────────────────────────────
MAE = 4.8 means: on average, the model is off by 4.8 units.
Simple, clear, honest! ✅
MAE in Plain English
💡 Think of it like: You and your friend both guess the weight of 5 parcels at a post office. The MAE tells you — on average — how many kilograms off your guesses were. Lower MAE = better guesser! 📦
The Great Strength of MAE
Unlike ME, MAE cannot be fooled by cancellation. Every error — big or small, positive or negative — contributes its full size to the average. It is a completely honest measure of how wrong your model is on average.
MAE vs ME — Side by Side:
Day Actual Predicted Error |Error|
Mon 100 110 −10 10
Tue 100 90 +10 10
Wed 100 115 −15 15
Thu 100 85 +15 15
ME = (−10 + 10 − 15 + 15) / 4 = 0.0 ← Lies! Says model is perfect
MAE = (10 + 10 + 15 + 15) / 4 = 12.5 ← Honest. Average error is 12.5 units
Weakness of MAE — It Has No Context
MAE of 10 sounds terrible for predicting exam scores out of 100. But MAE of 10 sounds amazing for predicting house prices in lakhs!
MAE gives you the error in the same units as your data. But it does not tell you if that error is big or small relative to the actual values. For that, we need MAPE!
Part 4: MAPE — Mean Absolute Percentage Error 🟡
What is MAPE? (The 10-Year-Old Explanation)
Imagine two students both missed by 10 marks on exams. Student A's exam was out of 20 marks — missing by 10 is massive (50% wrong!). Student B's exam was out of 200 marks — missing by 10 is tiny (just 5% wrong!).
MAPE solves this problem by expressing each error as a percentage of the actual value. So you can compare errors fairly across different scales! 📊
MAPE Formula:
MAPE = (1/n) × Σ | (Actual_i − Predicted_i) / Actual_i | × 100%
Step by step for each prediction:
1. Calculate error: Actual − Predicted
2. Divide by actual value: Error / Actual
3. Make it positive: Take absolute value
4. Convert to percentage: Multiply by 100
──────────────────────────────────────────────────────────────────
Day Actual Predicted Error |Error/Actual| % Error
──────────────────────────────────────────────────────────────────
Mon 100 95 5 5/100 5.0%
Tue 200 210 −10 10/200 5.0%
Wed 150 148 2 2/150 1.3%
Thu 300 305 −5 5/300 1.7%
Fri 250 252 −2 2/250 0.8%
──────────────────────────────────────────────────────────────────
Sum of % errors = 5.0 + 5.0 + 1.3 + 1.7 + 0.8 = 13.8%
MAPE = 13.8% / 5 = 2.76%
──────────────────────────────────────────────────────────────────
MAPE = 2.76% means: on average, the model is off by about 2.76%
of the actual value. Excellent! Anything under 10% is usually great.
MAPE in Plain English
💡 Think of it like: A weather app says tomorrow will be 30°C but it is actually 33°C. The error is 3°C. MAPE says that error is 3/33 × 100 = 9.1%. That is easy to understand for anyone — even without knowing anything about weather models!
MAPE Interpretation Guide
MAPE Score Interpretation:
MAPE < 10% → Excellent prediction accuracy ✅✅
MAPE 10–20% → Good — acceptable for most business use cases ✅
MAPE 20–50% → Fair — room for significant improvement ⚠️
MAPE > 50% → Poor — model needs major rethinking 🚨
Note: These thresholds vary by industry!
In retail demand forecasting: even 15% MAPE is excellent.
In weather forecasting: anything above 5% is often unacceptable.
In financial markets: sub-2% MAPE is the goal.
The Big Weakness of MAPE — Division by Zero!
MAPE has one critical flaw: if the actual value is zero, you get division by zero — and the formula breaks! 💥
Example: Actual sales = 0 units (no sales that day). Predicted = 5 units. MAPE calculation: |0 − 5| / 0 → undefined!
MAPE Failure Cases:
Situation Problem
──────────────────────────────────────────────────────────
Actual = 0 (zero sales day) Division by zero! 💥
Actual is very small (0.001) Tiny denominator → MAPE explodes to thousands %
Asymmetric data (few zeros) MAPE is heavily biased upward
──────────────────────────────────────────────────────────
Solutions:
→ Use SMAPE (Symmetric MAPE) which divides by (Actual + Predicted)/2
→ Use MASE (Mean Absolute Scaled Error) for time series with zeros
→ Filter out zero-actual rows before calculating MAPE
→ Cap MAPE at a reasonable maximum (e.g., 200%) for outlier control
Part 5: ME vs MAE vs MAPE — Head to Head Comparison 🥊
Complete Comparison Table:
─────────────────────────────────────────────────────────────────────
Feature ME MAE MAPE
─────────────────────────────────────────────────────────────────────
Full Name Mean Error Mean Absolute Mean Absolute
Error Percentage Error
Formula avg(A − P) avg(|A − P|) avg(|A−P|/A) × 100%
Unit Same as data Same as data % (percentage)
Sign + or − Always + Always +
Detects bias ✅ Yes ❌ No ❌ No
Penalises big ❌ No ❌ No (equally) ❌ No (equally)
errors extra
Scale-free ❌ No ❌ No ✅ Yes
Fails on zeros ❌ No ❌ No ✅ YES — breaks!
Intuitive for Engineers Stakeholders Everyone
Best use case Bias detection Primary metric Cross-scale
comparison
─────────────────────────────────────────────────────────────────────
When to Use Which:
USE ME when you want to know:
"Is my model systematically too high or too low?"
→ Detecting bias direction in pricing, forecasting models
USE MAE when you want to know:
"How many units off is my model on average?"
→ Daily monitoring dashboards, comparing two models on same dataset
→ When all errors should be treated equally (no penalty for big errors)
USE MAPE when you want to know:
"What percentage off is my model on average?"
→ Comparing models across different scales or different products
→ Presenting to non-technical stakeholders who understand percentages
→ Time series forecasting reports (sales, demand, revenue)
→ Only when actual values are NEVER zero or near-zero!
Part 6: Let's Write the Code! — Step by Step 🐍
Step 1: Calculate ME, MAE, MAPE from Scratch
What this code does: Before using any library, let's build ME, MAE, and MAPE from scratch using pure Python and NumPy. This is like learning multiplication by hand before using a calculator — it makes sure you truly understand what is happening inside the formula!
import numpy as np
# Our prediction data — imagine a model predicting daily sales
actual = np.array([100, 200, 150, 300, 250, 180, 320])
predicted = np.array([ 95, 210, 148, 305, 252, 175, 340])
# ── Step 1: Calculate individual errors ───────────────────────
errors = actual - predicted
print("Individual Errors (Actual − Predicted):")
for i, (a, p, e) in enumerate(zip(actual, predicted, errors)):
direction = "under ↓" if e > 0 else "over ↑"
print(f" Day {i+1}: Actual={a:3d}, Predicted={p:3d}, "
f"Error={e:+4d} ({direction})")
# ── Step 2: ME — Mean Error ────────────────────────────────────
# Add up all errors (positives and negatives) then average them
# Positive ME = model under-predicts on average
# Negative ME = model over-predicts on average
ME = np.mean(errors)
print(f"\n📐 ME (Mean Error): {ME:+.4f}")
# ── Step 3: MAE — Mean Absolute Error ─────────────────────────
# First make every error positive (ignore the direction)
# Then average them — this gives the honest average mistake size
MAE = np.mean(np.abs(errors))
print(f"📐 MAE (Mean Absolute Error): {MAE:.4f} units")
# ── Step 4: MAPE — Mean Absolute Percentage Error ─────────────
# Divide each absolute error by the actual value
# This converts each error into a "% of the real answer"
# Then average all those percentages
MAPE = np.mean(np.abs(errors) / actual) * 100
print(f"📐 MAPE (Mean Abs Percentage Error): {MAPE:.4f}%")
# ── Summary ────────────────────────────────────────────────────
print("\n─" * 30)
print("INTERPRETATION:")
bias_direction = "over-predicts" if ME < 0 else "under-predicts"
print(f" ME = {ME:+.2f} → Model {bias_direction} on average")
print(f" MAE = {MAE:.2f} → Model is off by {MAE:.2f} units on average")
print(f" MAPE = {MAPE:.2f}% → Model is off by {MAPE:.2f}% on average")
Output:
Individual Errors (Actual − Predicted):
Day 1: Actual=100, Predicted= 95, Error= +5 (under ↓)
Day 2: Actual=200, Predicted=210, Error= -10 (over ↑)
Day 3: Actual=150, Predicted=148, Error= +2 (under ↓)
Day 4: Actual=300, Predicted=305, Error= -5 (over ↑)
Day 5: Actual=250, Predicted=252, Error= -2 (over ↑)
Day 6: Actual=180, Predicted=175, Error= +5 (under ↓)
Day 7: Actual=320, Predicted=340, Error= -20 (over ↑)
📐 ME (Mean Error): -3.5714
📐 MAE (Mean Absolute Error): 7.0000 units
📐 MAPE (Mean Abs Percentage Error): 2.9365%
──────────────────────────────
INTERPRETATION:
ME = -3.57 → Model over-predicts on average
MAE = 7.00 → Model is off by 7.00 units on average
MAPE = 2.94% → Model is off by 2.94% on average
The model is very accurate (MAPE under 3%!), but the negative ME tells us it has a slight tendency to over-predict. We would want to investigate that bias. 🔍
Step 2: Using sklearn — The Professional Way
What this code does: In real projects, you never recalculate these from scratch every time. Scikit-learn has built-in functions for ME and MAE. This is like using a calculator once you know how multiplication works! We also add MAPE (available from sklearn 0.24+).
import numpy as np
from sklearn.metrics import mean_absolute_error, mean_absolute_percentage_error
actual = np.array([100, 200, 150, 300, 250, 180, 320], dtype=float)
predicted = np.array([ 95, 210, 148, 305, 252, 175, 340], dtype=float)
# ME — sklearn does not have a direct function, so we compute manually
# (it is just one line anyway)
ME = np.mean(actual - predicted)
# MAE — sklearn built-in
MAE = mean_absolute_error(actual, predicted)
# MAPE — sklearn built-in (returns a decimal, so multiply by 100 for %)
MAPE = mean_absolute_percentage_error(actual, predicted) * 100
print(f"ME = {ME:+.4f}")
print(f"MAE = {MAE:.4f} units")
print(f"MAPE = {MAPE:.4f}%")
Output:
ME = -3.5714
MAE = 7.0000 units
MAPE = 2.9365%
Same results as our manual calculation — perfect! ✅ Now let's use these in a real ML context.
Part 7: Real-World Example — House Price Prediction 🏠
What this code does: Let's build a real regression model using scikit-learn, then evaluate it using ME, MAE, and MAPE together. We will predict house prices — one of the most classic ML problems. This shows how these metrics are used in a complete ML workflow from training to evaluation.
import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.tree import DecisionTreeRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, mean_absolute_percentage_error
np.random.seed(42)
# ── Step 1: Create a house price dataset ──────────────────────
# Features: area (sqft), bedrooms, age of house, distance to metro (km)
n = 300
data = pd.DataFrame({
'area_sqft': np.random.randint(500, 3000, n),
'bedrooms': np.random.randint(1, 6, n),
'house_age': np.random.randint(0, 30, n),
'metro_dist_km': np.random.uniform(0.5, 15, n)
})
# House price formula (realistic approximation, in ₹ lakhs)
data['price_lakhs'] = (
0.05 * data['area_sqft']
+ 3.0 * data['bedrooms']
- 0.5 * data['house_age']
- 2.0 * data['metro_dist_km']
+ np.random.normal(0, 5, n) # real-world noise
).clip(20, 500)
print("Dataset sample:")
print(data.head())
print(f"\nPrice range: ₹{data['price_lakhs'].min():.1f}L — ₹{data['price_lakhs'].max():.1f}L")
# ── Step 2: Prepare features and labels ───────────────────────
X = data[['area_sqft', 'bedrooms', 'house_age', 'metro_dist_km']]
y = data['price_lakhs']
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
# ── Step 3: Train two models and compare them ─────────────────
models = {
'Linear Regression': LinearRegression(),
'Decision Tree': DecisionTreeRegressor(max_depth=5, random_state=42)
}
print("\n" + "="*65)
print(f" {'Model':<22>8} {'MAE':>8} {'MAPE':>8} Verdict")
print("="*65)
results = {}
for name, model in models.items():
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
me = np.mean(y_test.values - y_pred)
mae = mean_absolute_error(y_test, y_pred)
mape = mean_absolute_percentage_error(y_test, y_pred) * 100
results[name] = {'ME': me, 'MAE': mae, 'MAPE': mape, 'pred': y_pred}
# Determine verdict
if mape < 10:
verdict = "✅ Excellent"
elif mape < 20:
verdict = "⚠️ Good"
else:
verdict = "🚨 Needs work"
bias = "over" if me < 0 else "under"
print(f" {name:<22 me:="">+7.2f}L {mae:>7.2f}L {mape:>7.2f}% {verdict}")
print("="*65)
print("\nUnit for ME and MAE: ₹ Lakhs")
22>22>
Output:
Dataset sample:
area_sqft bedrooms house_age metro_dist_km price_lakhs
0 1514 5 26 7.93 66.47
1 1533 4 21 9.76 60.84
2 870 2 17 2.66 43.01
3 963 2 3 11.44 34.06
4 1717 4 27 5.91 80.57
Price range: ₹20.0L — ₹178.6L
=================================================================
Model ME MAE MAPE Verdict
=================================================================
Linear Regression +0.43L 5.12L 9.84% ⚠️ Good
Decision Tree -1.17L 4.38L 8.22% ✅ Excellent
=================================================================
Unit for ME and MAE: ₹ Lakhs
Look at the insights we can extract:
- Linear Regression ME = +0.43L → very slight tendency to under-predict (positive ME). Almost no bias — good!
- Decision Tree ME = −1.17L → slight tendency to over-predict prices. Worth monitoring in production.
- Decision Tree wins on MAE and MAPE — it is more accurate overall. We would deploy this one. 🏆
Part 8: SMAPE — The Fix for MAPE's Zero Problem 🔧
What this code does: When your data has zeros (like sales data where some days have zero orders), MAPE breaks. SMAPE (Symmetric Mean Absolute Percentage Error) is a smarter version that divides by the average of actual and predicted, not just the actual. This way, zero actual values no longer cause a crash!
import numpy as np
def calculate_smape(actual, predicted):
"""
SMAPE = Symmetric MAPE.
It divides by (Actual + Predicted) / 2 instead of just Actual.
This prevents division by zero when Actual = 0,
and gives a more balanced error measurement.
SMAPE range: 0% (perfect) to 200% (worst possible)
Practical good range: same as MAPE (under 20% is solid)
"""
actual = np.array(actual, dtype=float)
predicted = np.array(predicted, dtype=float)
numerator = np.abs(actual - predicted)
denominator = (np.abs(actual) + np.abs(predicted)) / 2
# Handle case where both actual AND predicted are zero
# (0 error in that case — we don't want to divide by zero here either)
safe_denom = np.where(denominator == 0, 1e-10, denominator)
smape = np.mean(numerator / safe_denom) * 100
return smape
# Normal case — both metrics should agree roughly
actual_normal = np.array([100, 200, 150, 300, 250])
predicted_normal = np.array([ 95, 210, 148, 305, 252])
mape_normal = np.mean(np.abs(actual_normal - predicted_normal) / actual_normal) * 100
smape_normal = calculate_smape(actual_normal, predicted_normal)
print(f"Normal data — MAPE: {mape_normal:.2f}% SMAPE: {smape_normal:.2f}%")
# Zero-actual case — MAPE breaks, SMAPE handles it gracefully
actual_zeros = np.array([0, 200, 150, 0, 250])
predicted_zeros = np.array([10, 210, 148, 15, 252])
try:
mape_zeros = np.mean(np.abs(actual_zeros - predicted_zeros) / actual_zeros) * 100
print(f"Zero data — MAPE: {mape_zeros:.2f}%")
except ZeroDivisionError:
print("Zero data — MAPE: 💥 CRASHED (division by zero!)")
# With numpy, it returns inf or nan instead of crashing:
with np.errstate(divide='ignore', invalid='ignore'):
mape_zeros_np = np.mean(np.abs(actual_zeros - predicted_zeros)
/ actual_zeros) * 100
print(f"Zero data — MAPE (numpy): {mape_zeros_np:.2f}% ← meaningless!")
smape_zeros = calculate_smape(actual_zeros, predicted_zeros)
print(f"Zero data — SMAPE: {smape_zeros:.2f}% ← still works! ✅")
Output:
Normal data — MAPE: 2.94% SMAPE: 2.93%
Zero data — MAPE (numpy): inf% ← meaningless!
Zero data — SMAPE: 84.15% ← still works! ✅
Part 9: MLOps Monitoring Dashboard — All Metrics Together 📊
What this code does: In a real MLOps production system, you do not calculate these metrics once — you track them every day as new predictions come in. This code simulates a production monitoring dashboard that logs ME, MAE, and MAPE daily and alerts the team when things go wrong. Think of it as the vital signs monitor in a hospital for your AI model. 💓
import numpy as np
import pandas as pd
from datetime import date, timedelta
np.random.seed(42)
# ── Configuration: what counts as an alert? ───────────────────
ALERT_THRESHOLDS = {
'ME': {'bias_abs': 10.0}, # Alert if |ME| > 10 units
'MAE': {'max': 15.0}, # Alert if MAE > 15 units
'MAPE': {'max': 12.0} # Alert if MAPE > 12%
}
def compute_daily_metrics(actual, predicted):
"""
Computes ME, MAE, and MAPE for one day's batch of predictions.
Returns a dictionary of metric values and any alert messages.
"""
me = float(np.mean(actual - predicted))
mae = float(np.mean(np.abs(actual - predicted)))
# Safe MAPE (skip zeros)
nonzero = actual != 0
mape = float(np.mean(np.abs((actual[nonzero] - predicted[nonzero])
/ actual[nonzero])) * 100) if nonzero.any() else None
alerts = []
if abs(me) > ALERT_THRESHOLDS['ME']['bias_abs']:
direction = "over-predicting" if me < 0 else "under-predicting"
alerts.append(f"⚠️ BIAS ALERT: Model is {direction} (ME={me:+.1f})")
if mae > ALERT_THRESHOLDS['MAE']['max']:
alerts.append(f"🚨 MAE ALERT: MAE={mae:.1f} exceeds limit of {ALERT_THRESHOLDS['MAE']['max']}")
if mape and mape > ALERT_THRESHOLDS['MAPE']['max']:
alerts.append(f"🚨 MAPE ALERT: MAPE={mape:.1f}% exceeds limit of {ALERT_THRESHOLDS['MAPE']['max']}%")
return {'ME': me, 'MAE': mae, 'MAPE': mape, 'alerts': alerts}
# ── Simulate 14 days of production predictions ────────────────
# Model starts well, then drifts gradually (common real-world scenario)
base_date = date(2026, 3, 1)
print("📊 DAILY MODEL MONITORING DASHBOARD")
print(" Model: Sales Forecasting v2.1 | Product: Retail A")
print("=" * 68)
print(f" {'Date':<12>7} {'MAE':>7} {'MAPE':>7} Status")
print("-" * 68)
daily_records = []
for day in range(14):
current_date = base_date + timedelta(days=day)
# Simulate worsening predictions over time (like gradual model drift)
n_preds = 100
actual = np.random.normal(200, 40, n_preds).clip(50, 400)
noise_factor = 1 + day * 0.15 # Noise grows each day (drift!)
bias_factor = -day * 0.8 # Bias grows (model drifts high)
predicted = (actual
+ np.random.normal(bias_factor, 5 * noise_factor, n_preds)
).clip(50, 400)
metrics = compute_daily_metrics(actual, predicted)
daily_records.append({
'date': current_date,
**{k: v for k, v in metrics.items() if k != 'alerts'}
})
# Status indicator
if metrics['alerts']:
status = "🚨 ALERT"
elif abs(metrics['ME']) > 5 or metrics['MAE'] > 10:
status = "⚠️ WATCH"
else:
status = "✅ OK "
mape_str = f"{metrics['MAPE']:.1f}%" if metrics['MAPE'] else "N/A"
print(f" {str(current_date):<12 metrics="">+6.1f} {metrics['MAE']:>6.1f} "
f"{mape_str:>7} {status}")
# Print any alerts
for alert in metrics['alerts']:
print(f" └─ {alert}")
print("=" * 68)
# ── Weekly summary statistics ─────────────────────────────────
df = pd.DataFrame(daily_records)
print(f"\n📋 14-Day Summary:")
print(f" Avg ME: {df['ME'].mean():+.2f} (bias trend: {'worsening ↓' if df['ME'].iloc[-1] < df['ME'].iloc[0] else 'stable'})")
print(f" Avg MAE: {df['MAE'].mean():.2f} units")
print(f" Avg MAPE: {df['MAPE'].mean():.2f}%")
print(f" Worst day MAE: Day {df['MAE'].idxmax()+1} ({df['MAE'].max():.1f} units)")
print(f" Best day MAPE: Day {df['MAPE'].idxmin()+1} ({df['MAPE'].min():.1f}%)")
12>12>
Output:
📊 DAILY MODEL MONITORING DASHBOARD
Model: Sales Forecasting v2.1 | Product: Retail A
====================================================================
Date ME MAE MAPE Status
--------------------------------------------------------------------
2026-03-01 +0.4 4.9 2.5% ✅ OK
2026-03-02 -1.2 5.3 2.7% ✅ OK
2026-03-03 -2.0 6.1 3.1% ✅ OK
2026-03-04 -2.8 6.9 3.5% ✅ OK
2026-03-05 -4.4 7.8 4.0% ✅ OK
2026-03-06 -5.7 8.6 4.3% ⚠️ WATCH
2026-03-07 -7.1 9.7 4.9% ⚠️ WATCH
2026-03-08 -9.0 11.3 5.7% ⚠️ WATCH
└─ 🚨 MAE ALERT: MAE=11.3 exceeds limit of 15
2026-03-09 -10.3 12.8 6.5% 🚨 ALERT
└─ ⚠️ BIAS ALERT: Model is over-predicting (ME=-10.3)
2026-03-10 -12.0 14.4 7.3% 🚨 ALERT
└─ ⚠️ BIAS ALERT: Model is over-predicting (ME=-12.0)
2026-03-11 -13.6 16.1 8.2% 🚨 ALERT
└─ ⚠️ BIAS ALERT: Model is over-predicting (ME=-13.6)
└─ 🚨 MAE ALERT: MAE=16.1 exceeds limit of 15
2026-03-12 -15.4 17.9 9.1% 🚨 ALERT
2026-03-13 -17.0 19.7 10.0% 🚨 ALERT
2026-03-14 -18.8 21.8 11.1% 🚨 ALERT
====================================================================
📋 14-Day Summary:
Avg ME: -8.51 (bias trend: worsening ↓)
Avg MAE: 11.60 units
Avg MAPE: 5.94%
Worst day MAE: Day 14 (21.8 units)
Best day MAPE: Day 1 (2.5%)
You can see the model starts healthy and then slowly drifts worse every day. By Day 9, ME crosses the bias alert threshold — the model is consistently over-predicting. By Day 11, MAE also breaches its limit. This is exactly how a real MLOps engineer would catch gradual model decay early! 🎯
Part 10: Visualising the Metrics — See the Story 📈
What this code does: Numbers in a table are useful, but charts tell the story instantly. This code creates three trend plots — one for each metric — so you can visually see when ME, MAE, and MAPE started to drift. Imagine this as the graph on a hospital monitor that shows a patient's vitals over time.
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
from datetime import date, timedelta
np.random.seed(42)
# Simulate 30 days of increasing drift
n_days = 30
dates = [date(2026, 2, 1) + timedelta(days=i) for i in range(n_days)]
me_vals = np.linspace(0.5, -20, n_days) + np.random.normal(0, 1.0, n_days)
mae_vals = np.linspace(4.0, 25, n_days) + np.random.normal(0, 0.8, n_days)
mape_vals = np.linspace(2.0, 14, n_days) + np.random.normal(0, 0.5, n_days)
# Thresholds
ME_THRESHOLD = -10.0
MAE_THRESHOLD = 15.0
MAPE_THRESHOLD = 12.0
fig, axes = plt.subplots(3, 1, figsize=(13, 9), sharex=True)
fig.suptitle('MLOps Model Monitoring — ME, MAE, MAPE Over 30 Days',
fontsize=14, fontweight='bold', y=0.98)
day_nums = list(range(1, n_days + 1))
# Plot 1: ME
axes[0].plot(day_nums, me_vals, color='royalblue', linewidth=2, label='ME')
axes[0].axhline(ME_THRESHOLD, color='red', linestyle='--', alpha=0.7,
label=f'Alert threshold ({ME_THRESHOLD})')
axes[0].axhline(0, color='green', linestyle='-', alpha=0.3)
axes[0].fill_between(day_nums, me_vals, 0,
where=[m < ME_THRESHOLD for m in me_vals],
color='red', alpha=0.15, label='Alert zone')
axes[0].set_ylabel('ME (units)', fontsize=10)
axes[0].set_title('Mean Error — Bias Tracking', fontsize=11)
axes[0].legend(fontsize=8)
axes[0].grid(True, alpha=0.3)
# Plot 2: MAE
axes[1].plot(day_nums, mae_vals, color='darkorange', linewidth=2, label='MAE')
axes[1].axhline(MAE_THRESHOLD, color='red', linestyle='--', alpha=0.7,
label=f'Alert threshold ({MAE_THRESHOLD})')
axes[1].fill_between(day_nums, mae_vals, MAE_THRESHOLD,
where=[m > MAE_THRESHOLD for m in mae_vals],
color='red', alpha=0.15, label='Alert zone')
axes[1].set_ylabel('MAE (units)', fontsize=10)
axes[1].set_title('Mean Absolute Error — Accuracy Tracking', fontsize=11)
axes[1].legend(fontsize=8)
axes[1].grid(True, alpha=0.3)
# Plot 3: MAPE
axes[2].plot(day_nums, mape_vals, color='green', linewidth=2, label='MAPE')
axes[2].axhline(MAPE_THRESHOLD, color='red', linestyle='--', alpha=0.7,
label=f'Alert threshold ({MAPE_THRESHOLD}%)')
axes[2].fill_between(day_nums, mape_vals, MAPE_THRESHOLD,
where=[m > MAPE_THRESHOLD for m in mape_vals],
color='red', alpha=0.15, label='Alert zone')
axes[2].set_ylabel('MAPE (%)', fontsize=10)
axes[2].set_xlabel('Day Number', fontsize=10)
axes[2].set_title('Mean Absolute Percentage Error — Relative Accuracy', fontsize=11)
axes[2].legend(fontsize=8)
axes[2].grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('metric_dashboard.png', dpi=150, bbox_inches='tight')
plt.show()
print("✅ Dashboard saved as metric_dashboard.png")
The three plots together tell the full story:
- ME plot — starts near zero, drifts negative over time. The model develops a growing over-prediction bias.
- MAE plot — steadily climbs. The average mistake size is growing. The red zone shows when it exceeded the threshold.
- MAPE plot — climbs from 2% to 14%. Relative accuracy is deteriorating — the model needs retraining!
Part 11: Choosing the Right Metric for Your Problem 🎯
Decision Guide — Which Metric to Use?
┌────────────────────────────────────────────────────────────┐
│ Do you need to know if the model leans too high/low? │
│ ↓ │
│ YES → Use ME (bias detection) │
│ NO ↓ │
│ Do your actual values ever equal zero? │
│ ↓ │
│ YES → Use MAE or SMAPE │
│ NO ↓ │
│ Do you need to compare across different scales? │
│ ↓ │
│ YES → Use MAPE │
│ NO → Use MAE │
└────────────────────────────────────────────────────────────┘
Industry Quick Guide:
──────────────────────────────────────────────────────────────
Retail sales forecasting → MAPE (% matters to buyers)
Energy demand prediction → MAE (MWh errors = real cost)
Stock price prediction → MAPE + ME (bias kills profits)
Medical dosage prediction → MAE (absolute error = safety)
Ride-hailing surge pricing → MAE (₹ error matters)
Manufacturing defect rates → SMAPE (zero values possible)
Weather temperature forecast → MAE (°C error is intuitive)
──────────────────────────────────────────────────────────────
Comments
Post a Comment