C.W.K.
Stream
Lesson 08 of 10 · published

The Hidden Cost of ML

~28 min · mlops, lifecycle

Level 0Scout
0 XP0/48 lessons0/11 achievements
0/120 XP to next level120 XP to go0% complete

Training is the easy part

The first model takes a week. The next year takes the rest of the team. Most ML costs live outside training: labeling pipelines, feature pipelines, drift monitoring, retraining cadence, rollback procedures, on-call, alerting, and the documentation that lets a new engineer ship a fix without paging the original author.

The lifecycle bill

  • Data — collection, cleaning, labeling, ongoing quality checks.
  • Features — pipeline code, freshness SLAs, backfill stories.
  • Training — compute, experimentation, hyperparameter search.
  • Serving — latency, autoscaling, A/B framework, fallback path.
  • Monitoring — input drift, output drift, calibration, business KPIs.
  • Governance — model cards, audits, approvals, deprecation plan.

The honest plan

Estimate each line item before you commit to ML. If the team cannot staff monitoring, the model will silently degrade. If there is no rollback path, the first bad day becomes the last good week.

Code

A minimal lifecycle budget·python
lifecycle_budget = {
    "data_labeling_hrs_per_month": 40,
    "feature_pipeline_oncall": "shared rotation",
    "training_compute_usd_per_month": 250,
    "serving_latency_p99_ms": 150,
    "retraining_cadence": "weekly",
    "rollback_target_minutes": 10,
    "monitoring_dashboards": ["input_drift", "output_drift", "business_kpi"],
}
A rollback hatch is part of the model·python
def serve(features, primary, fallback, allow_primary):
    if allow_primary():
        try:
            return primary.predict(features)
        except Exception as exc:
            log("primary failed, falling back", exc=exc)
    return fallback.predict(features)

External links

Exercise

Sketch the lifecycle budget for an ML idea your team is considering. Include labeling hours per month, retraining cadence, rollback time target, and which dashboards must exist before launch. Compare to your team's actual capacity.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.