Prediction, Explanation, and Decision-Making

Three jobs that get conflated

Stakeholders often ask for a model when they actually want one of three different things: a prediction (what number, what label), an explanation (why this prediction), or a decision (what action). Each job has a different success metric and a different deployment shape.

Why this matters

An accurate predictor can still be a bad decision tool if the threshold is wrong, the action is misaligned, or the user cannot tolerate the false-positive cost. A great explanation can still be a bad decision tool if the explanation does not point to a feasible intervention. A confident decision system without inspection is a liability when it errs.

Explanation depends on the audience

An engineer debugging the model needs a different explanation from a customer challenging a decision. Feature importance can show what the model relied on, but it does not prove that changing that feature will improve the outcome. If the intended output is advice, distinguish predictive evidence from a feasible causal intervention.

Designing for the right job

Write the job sentence first: given X, predict Y, then take action Z when probability exceeds threshold T. If the team cannot finish that sentence, you are not ready to model. The threshold and action belong on the design doc, not in the deploy script as a magic number.

Log the boundaries separately

Record the raw score, threshold, policy version, final action, and model version as separate fields. When an outcome is bad, this lets you ask whether ranking failed, the threshold reflected the wrong cost, or the action itself was ineffective. Calling all five things “the model” erases that accountability.

Code

Separate the prediction from the decision·python

probs = model.predict_proba(X)[:, 1]

def decide(prob, threshold=0.42, cost_fp=5, cost_fn=50):
    expected_cost_action = cost_fp * (1 - prob)
    expected_cost_skip = cost_fn * prob
    return "act" if expected_cost_action < expected_cost_skip else "skip"

Log enough for explanation later·python

import json, datetime

def log_decision(user_id, features, prob, decision):
    payload = {
        "ts": datetime.datetime.utcnow().isoformat(),
        "user_id": user_id,
        "prob": float(prob),
        "decision": decision,
        "features": features,
        "model_version": "churn-2026-05-03",
    }
    print(json.dumps(payload))

Three jobs that get conflated

Why this matters

Explanation depends on the audience

Designing for the right job

Log the boundaries separately

Code

External links

Exercise

Progress

Comments 0