How do you handle severe class imbalance in a production AWS ML system?

Updated Aug 1, 2026

Short answer

Choose metrics that survive imbalance (PR-AUC, recall at fixed precision), use class weighting rather than naive oversampling, tune the decision threshold against business cost rather than defaulting to 0.5, and monitor the positive rate in production.

Deep explanation

At 0.1% positives, a model predicting "negative" always achieves 99.9% accuracy. Every part of the pipeline must be built around that fact.

1. Metrics. ROC-AUC is misleading under heavy imbalance because the false-positive rate is computed against a huge negative class, so large absolute increases in false positives barely move it. Use:

PR-AUC — sensitive to performance on the minority class.
Recall at fixed precision — the operational framing: "what fraction of fraud do we catch while keeping the review queue at 5% false positives?"

**2.…

Unlock with a Pro subscription to view this section.

View pricing