Pipeline Artifact

~26 min · artifact, serialization, deployment

Level 0Scout

0 XP0/48 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

The artifact is the contract

The deployable thing is not the model. It is the entire pipeline (preprocessing + model) saved as a single artifact. The discipline is to load the artifact in a fresh environment and run predict on a raw row without any extra glue. If that does not work, the pipeline is not really an artifact yet.

What goes inside

The fitted Pipeline (preprocessing + model).
The list of expected input column names and dtypes.
The model version, training timestamp, code commit, dataset hash.
The chosen threshold (if classification) and the metric value at training time.

Format choices

joblib is the sklearn default. onnx is the cross-language option for serving in non-Python runtimes. For boosting libraries, save in their native format too (model.save_model('booster.json')) so you can inspect splits later without unpickling.

Code

Save the pipeline plus its metadata·python

import json, joblib, hashlib, datetime

joblib.dump(pipe, "artifacts/churn_v1.joblib")
metadata = {
    "version": "churn_v1",
    "trained_at": datetime.datetime.utcnow().isoformat(),
    "git_commit": git_sha,
    "data_hash": hashlib.md5(open("data/processed/train.parquet", "rb").read()).hexdigest(),
    "features": list(X_train.columns),
    "threshold": 0.42,
    "pr_auc_train": float(pr_auc_train),
}
with open("artifacts/churn_v1.json", "w") as f:
    json.dump(metadata, f, indent=2)

Load and validate before serving·python

import joblib, json
import pandas as pd

pipe = joblib.load("artifacts/churn_v1.joblib")
meta = json.load(open("artifacts/churn_v1.json"))

def predict(raw_row: dict) -> dict:
    X = pd.DataFrame([raw_row])
    missing = set(meta["features"]) - set(X.columns)
    assert not missing, f"missing features: {missing}"
    p = pipe.predict_proba(X[meta["features"]])[0, 1]
    return {"prob": float(p), "label": int(p >= meta["threshold"]), "version": meta["version"]}

External links

Exercise

Save your trained pipeline as a joblib artifact plus a metadata JSON. From a fresh shell, write a 20-line script that loads both and serves predictions for a raw row. Verify the prediction matches the training notebook.