Where Deep Learning Is Overkill

Four red flags

Tiny tabular data with strong features. A few thousand rows, a few dozen well-named columns, a clear target — gradient boosting will eat your transformer for breakfast and you will not have to argue about VRAM.

A simple rule already works. If if user.age < 18 then block solves the business problem, do not propose a model. The rule is auditable, instant, free, and impossible to overfit.

Hard interpretability requirements. Some domains (credit scoring, medical triage, regulated industries) require a person to defend a decision. A linear model with monotonic constraints is honest. A black-box transformer is a lawsuit.

Real-time latency budgets in milliseconds with no GPUs. A 100ms p99 latency on CPU does not have room for a transformer forward pass. Quantization and distillation help, but a tuned tree model often hits the spec without any of that work.

Warning: The most expensive deep-learning project is the one that ships, almost works, and replaces an honest baseline that worked. The team is now on the hook for incident response on a system they cannot fully reason about.

The serious-grown-up question

Before reaching for a neural network, write the answer to: what is the simplest thing that could possibly work, and what does it score? If the simplest thing scores 0.78 and you need 0.84, the room for deep learning is six points. If the simplest thing scores 0.95, the room is closed.

Principle: Choosing not to use deep learning is a senior engineering decision. The hype cycle rewards the opposite, which is exactly why doing the boring thing on purpose builds career-grade trust.

Code

Honest baselines beat hopeful neural nets·python

import time
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score

t0 = time.time()
lr = LogisticRegression(max_iter=2000, C=1.0).fit(X_train, y_train)
t_train = time.time() - t0

t0 = time.time()
preds = lr.predict_proba(X_val)[:, 1]
t_infer = time.time() - t0

print(f"AUROC: {roc_auc_score(y_val, preds):.3f}")
print(f"Train time: {t_train:.2f}s   Inference time: {t_infer*1000:.1f}ms")
print(f"Inference latency / row: {t_infer / len(X_val) * 1e6:.1f} microseconds")

Four red flags

The serious-grown-up question

Code

External links

Exercise

Progress

Comments 0