Feature Importance

~28 min · importance, interpretation

Level 0Scout

0 XP0/48 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

Three flavors of importance

Built-in tree importance — fast but biased toward high-cardinality features.
Permutation importance — model-agnostic; shuffles a feature and measures the drop in score. Honest but expensive.
SHAP values — per-prediction contribution; expensive but powerful for local explanations.

What importance is and is not

Importance measures predictive contribution under the current data distribution. It is not causality. It is not stability under intervention. A feature can be highly important and still be wrong to act on, because the action would change the distribution.

Communicating it well

Show the top-10 with absolute scores, plus a SHAP summary plot for shape (positive vs negative direction). Always include a sanity check column the team trusts; if your model says it does not matter and the team knows it does, your data or framing is wrong.

Code

Permutation importance as the default·python

from sklearn.inspection import permutation_importance

result = permutation_importance(
    model, X_val, y_val, n_repeats=20, random_state=7,
    scoring="average_precision", n_jobs=-1
)
for name, score in sorted(zip(X_val.columns, result.importances_mean), key=lambda kv: -kv[1])[:10]:
    print(f"{score:+.4f}  {name}")

SHAP for local explanation·python

import shap

explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_val.sample(500, random_state=7))
shap.summary_plot(shap_values, X_val.sample(500, random_state=7))

External links

Exercise

Compute permutation importance and SHAP summary for your model. Cross-check the top-10 against your team's expectations. Investigate the biggest disagreement before shipping.