Society of Actuaries (SOA) PA Practice Exam

How does a model with an AUC score below 0.5 function?

1. It makes predictions worse than random selection

2. It is perfectly well-calibrated

3. It indicates a balanced classification model

4. It always predicts the correct outcomes

The correct answer is: It makes predictions worse than random selection

A model with an AUC score below 0.5 indeed indicates that it is performing worse than random selection. The AUC, or Area Under the Curve, is a measure used in binary classification to evaluate how well the model distinguishes between the positive and negative classes. An AUC of 0.5 suggests that the model is no better than random chance; in fact, an AUC below 0.5 implies that the model is systematically misclassifying the outcomes. When the AUC falls below 0.5, it means that, when given any two randomly selected instances, the model is more likely to assign a higher score to the negative instance than to the positive one, indicating the model has inverted the true positive and negative predictions. Therefore, this outcome confirms that the model is fundamentally flawed in its ability to classify the data appropriately, making it worse than random guessing. Other choices do not accurately describe this scenario: - A model being perfectly well-calibrated, regardless of an AUC score, suggests it properly reflects the probabilities of the outcomes, not necessarily tied to misclassification. - An indication of a balanced classification model would not be associated with an AUC below 0.5, as balance refers to the model