Skip to content
C.W.K.
Stream
Lesson 02 of 05 · published

Feature Importance

~28 min · importance, interpretation

Level 0Scout
0 XP0/48 lessons0/11 achievements
0/120 XP to next level120 XP to go0% complete

Importance asks what this model relies on

Feature importance describes a particular fitted model under a particular data distribution and metric. It does not reveal a variable's intrinsic value. State model, evaluation set, method, and scoring rule whenever publishing a ranking.

Tree-internal importance is a fast first look

Impurity importance sums the gain from tree splits. It is convenient but derived from training behavior and can favor continuous or high-cardinality features with many possible split points.

Permutation importance measures score dependence

Shuffle one feature on held-out data and measure the metric drop. This is model-agnostic, but unrealistic shuffles can create impossible rows, and a correlated substitute can make an important feature appear dispensable. Repeat the shuffle and report uncertainty.

SHAP allocates one prediction relative to a baseline

SHAP values distribute the difference between a prediction and a reference value under background and feature-dependence assumptions. They can support local explanations, but the baseline, encoded value, units, and missingness state must be shown.

Direction and shape require their own view

An absolute importance bar does not show whether larger values raise or lower a score or whether the relationship bends. Use appropriate dependence or attribution summaries and inspect raw cases, while remembering that the shape describes model behavior rather than a guaranteed real-world law.

Correlated features share and steal credit

When columns contain similar information, a model may substitute one for another. Group related features, remove them together in ablations, and repeat measurements across folds or seeds instead of trusting one brittle top-ten order.

Importance is not intervention effect

A positive contribution does not mean changing the feature will improve the outcome. Geography or access proxies may be predictive and still be inappropriate levers. Importance and local explanations account for a model calculation, not causality.

Use surprises as audit opportunities

Pair global summaries with held-out permutation results, slice metrics, and audited local examples. Random noise should not remain important, removing target leakage should collapse performance, and shuffled labels should destroy the signal. When domain expectations disagree, investigate both data and expectations rather than declaring either side correct.

Code

Permutation importance as the default·python
from sklearn.inspection import permutation_importance

result = permutation_importance(
    model, X_val, y_val, n_repeats=20, random_state=7,
    scoring="average_precision", n_jobs=-1
)
for name, score in sorted(zip(X_val.columns, result.importances_mean), key=lambda kv: -kv[1])[:10]:
    print(f"{score:+.4f}  {name}")
SHAP for local explanation·python
import shap

explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_val.sample(500, random_state=7))
shap.summary_plot(shap_values, X_val.sample(500, random_state=7))

External links

Exercise

Compute permutation importance and SHAP summary for your model. Cross-check the top-10 against your team's expectations. Investigate the biggest disagreement before shipping.

Progress

Progress is local-only — sign in to sync across devices.
Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)
Sign inPlease sign in to comment.

No comments yet — be the first.