Lightweight Deployment

~30 min · deployment, fastapi

Level 0Scout

0 XP0/48 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

You probably do not need Kubernetes

For most classical ML use cases, a single FastAPI server behind a load balancer is enough. The model is a few hundred MB; latency is a few ms; the team is two engineers. Reach for heavier infrastructure only when traffic, latency, or compliance demand it.

The minimum surface area

An HTTP endpoint that accepts a raw row, returns probability + decision + version.
A health endpoint for the load balancer.
A version endpoint that returns model metadata.
Structured logging of every prediction (input, output, version, latency).
A circuit breaker to the rule-based fallback when the model fails.

The deployment ritual

Shadow-deploy first: route 10% of traffic to the new model, log predictions but use the old model's decision. Compare distributions. Promote only after the shadow passes. Always keep the previous artifact one click away for rollback.

Code

Minimal FastAPI server for a sklearn pipeline·python

import joblib, json
import pandas as pd
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()
pipe = joblib.load("artifacts/churn_v1.joblib")
meta = json.load(open("artifacts/churn_v1.json"))

class Row(BaseModel):
    payload: dict

@app.get("/health")
def health():
    return {"ok": True, "version": meta["version"]}

@app.post("/score")
def score(row: Row):
    X = pd.DataFrame([row.payload])[meta["features"]]
    p = float(pipe.predict_proba(X)[0, 1])
    return {"prob": p, "label": int(p >= meta["threshold"]), "version": meta["version"]}

Structured prediction logging·python

import json, datetime

def log_prediction(payload, prob, label, version, latency_ms):
    print(json.dumps({
        "ts": datetime.datetime.utcnow().isoformat(),
        "version": version,
        "latency_ms": latency_ms,
        "prob": prob,
        "label": label,
        "payload": payload,
    }))

External links

Exercise

Wrap your saved pipeline in a 30-line FastAPI app. Hit /score with a raw row using curl. Verify the response includes probability, label, and model version. Add structured logging of every request.