ROC-AUC vs PR-AUC

~28 min · metrics, roc, pr-auc

Level 0Scout

0 XP0/48 lessons0/11 achievements

0/120 XP to next level120 XP to go0% complete

Two threshold-free summaries

ROC-AUC summarizes how well the model orders positives above negatives, integrated over every possible threshold. PR-AUC (average precision) summarizes the precision/recall curve. Both ignore the threshold; both depend on the score, not the label.

When to use which

Use ROC-AUC when classes are roughly balanced and you want a single comparison number across runs. Use PR-AUC when the positive class is rare, because ROC-AUC can stay deceptively high even when the model rarely catches positives. For fraud, churn, and anomaly detection, PR-AUC is almost always the right choice.

The 0.5 myth

An ROC-AUC of 0.5 means the model is no better than random ordering. PR-AUC's random baseline is the positive class prevalence; if 1% of users churn, random PR-AUC is 0.01, not 0.5. Always report PR-AUC alongside the prevalence.

Code

Both AUCs side by side·python

from sklearn.metrics import roc_auc_score, average_precision_score

roc = roc_auc_score(y_val, probs)
pr = average_precision_score(y_val, probs)
prevalence = y_val.mean()
print(f"ROC-AUC={roc:.3f}  PR-AUC={pr:.3f}  prevalence={prevalence:.3f}")

Plot the PR curve to see where the model breaks·python

from sklearn.metrics import PrecisionRecallDisplay
import matplotlib.pyplot as plt

PrecisionRecallDisplay.from_predictions(y_val, probs)
plt.title("Validation precision-recall curve")
plt.grid(True)
plt.show()

External links

Exercise

On your classifier, compute and report ROC-AUC, PR-AUC, and the positive-class prevalence in one line. If your team uses ROC-AUC by default and the positive class is rare, propose switching to PR-AUC and explain why in two sentences.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousConfusion Matrix and Metrics Next →Threshold Tuning and Business Cost

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.