Good and Bad ML Use Cases

Where ML earns its keep

ML is a strong fit when the pattern is real but too complex for hand-coded rules, when labels are abundant and reliable, when mistakes are recoverable or low-stakes, and when the data distribution does not shift faster than you can retrain. Tabular ranking, fraud scoring, demand forecasting, recommendations, and image triage are classic wins.

Where ML is a bad first choice

The rule already works and is auditable. Replacing it adds risk for no gain.
Mistakes are catastrophic and irreversible. Do not let an ML score be the only safety net.
Labels are scarce, ambiguous, or contested. The model will encode the disagreement.
The world drifts faster than you can ship a new model.
The decision is regulated and requires a human reason for every outcome.

Compare the non-ML alternatives

Improving search, changing an input form, automating a stable rule, or giving a human a better priority queue can create more value with less risk. Compare accuracy, build time, auditability, rollback, and staffing rather than treating “no model” as doing nothing. A slightly weaker rule may win after lifecycle cost is counted.

Triaging an idea in five minutes

For each ML idea, score four dimensions on a 1-5 scale: pattern complexity, label availability, mistake reversibility, and drift speed. Anything with two or more scores below 3 needs a non-ML alternative or a much smaller scope before you commit to a model. The score is a conversation aid, not proof; validate a promising idea against a simple baseline and a small field test.

Code

A five-minute ML fit checklist·python

def ml_fit_score(idea):
    pattern = idea["pattern_complexity"]
    labels = idea["label_availability"]
    reversibility = idea["mistake_reversibility"]
    drift = 6 - idea["drift_speed"]  # invert: slower drift is better
    return (pattern + labels + reversibility + drift) / 4

fraud = {"pattern_complexity": 5, "label_availability": 4, "mistake_reversibility": 4, "drift_speed": 4}
print(ml_fit_score(fraud))  # ~3.75 → strong fit

Ship the simplest baseline first·python

from sklearn.dummy import DummyClassifier

baseline = DummyClassifier(strategy="prior").fit(X_tr, y_tr)
print("baseline accuracy:", baseline.score(X_te, y_te))

Where ML earns its keep

Where ML is a bad first choice

Compare the non-ML alternatives

Triaging an idea in five minutes

Code

External links

Exercise

Progress

Comments 0