Importance asks what this model relies on
Feature importance describes a particular fitted model under a particular data distribution and metric. It does not reveal a variable's intrinsic value. State model, evaluation set, method, and scoring rule whenever publishing a ranking.
Tree-internal importance is a fast first look
Impurity importance sums the gain from tree splits. It is convenient but derived from training behavior and can favor continuous or high-cardinality features with many possible split points.
Permutation importance measures score dependence
Shuffle one feature on held-out data and measure the metric drop. This is model-agnostic, but unrealistic shuffles can create impossible rows, and a correlated substitute can make an important feature appear dispensable. Repeat the shuffle and report uncertainty.
SHAP allocates one prediction relative to a baseline
SHAP values distribute the difference between a prediction and a reference value under background and feature-dependence assumptions. They can support local explanations, but the baseline, encoded value, units, and missingness state must be shown.
Direction and shape require their own view
An absolute importance bar does not show whether larger values raise or lower a score or whether the relationship bends. Use appropriate dependence or attribution summaries and inspect raw cases, while remembering that the shape describes model behavior rather than a guaranteed real-world law.
Correlated features share and steal credit
When columns contain similar information, a model may substitute one for another. Group related features, remove them together in ablations, and repeat measurements across folds or seeds instead of trusting one brittle top-ten order.
Importance is not intervention effect
A positive contribution does not mean changing the feature will improve the outcome. Geography or access proxies may be predictive and still be inappropriate levers. Importance and local explanations account for a model calculation, not causality.
Use surprises as audit opportunities
Pair global summaries with held-out permutation results, slice metrics, and audited local examples. Random noise should not remain important, removing target leakage should collapse performance, and shuffled labels should destroy the signal. When domain expectations disagree, investigate both data and expectations rather than declaring either side correct.