Society of Actuaries (SOA) PA Practice Exam

What is a major feature of Random Forests in reducing overfitting?

1. They increase the number of predictors used.

2. They are less likely to use bootstrap samples.

3. They use bagging and random feature selection for each tree.

4. They solely focus on reducing bias in predictions.

The correct answer is: They use bagging and random feature selection for each tree.

The correct answer centers on the method of using bagging and random feature selection for each tree, which is a significant aspect of how Random Forests mitigate the problem of overfitting. Random Forests employ a technique called bootstrap aggregating, or bagging, which involves creating multiple subsets of the training data by sampling with replacement. Each of these subsets is then used to train individual decision trees. This process helps to ensure that each tree is trained on a slightly different dataset, which promotes diversity among the trees and reduces the model's overall variance. In addition to bagging, Random Forests also incorporate random feature selection when building each tree. Instead of considering all available features for splitting at each node, a random subset of features is chosen. This randomness prevents any single feature from dominating the model, thereby reducing the likelihood of overfitting to noise specific to the training data. Together, these strategies help to create a more robust model that generalizes better to unseen data, striking a good balance between capturing significant patterns and avoiding the pitfalls of overfitting. Through the combination of bagging and random feature selection, Random Forests are able to create a diverse set of trees that collectively lead to a more stable and accurate prediction.