Ensemble learning is an effective technique in supervised machine learning that takes multiple trained classifiers and combines their prediction into one. Numerous studies have demonstrated that ensembles consistently outperform individual classifiers across a wide range of tasks. However, simply aggregating all available models is often inefficient and may even degrade performance due to redundancy among similar models. Selecting an intelligent subset of models from a larger pool can therefore yield ensembles that are both more accurate and computationally efficient. This is referred to as ensemble selection or ensemble pruning. Finding the optimal ensemble configuration – specifically, the types and number of classifiers to include – is a non-trivial problem, particularly in heterogeneous ensembles, which consist of different model types. While early research commonly relied on manual or heuristic selection of ensemble members, methods have emerged that attempt to determine the ideal model configuration and ensemble size automatically.

This thesis identifies and categorizes a broad set of ensemble selection approaches. From this taxonomy, four methods for the automated selection of the classifier number in heterogeneous ensembles are selected for detailed empirical comparison: Greedy Ensemble Selection (GES), Directed Hill Climbing Ensemble Pruning using Uncertainty Weighted Accuracy (DHCEP using UWA), a Clustering-based Ensemble Selection approach, and Ant Colony Optimization-Stacking (ACO-Stacking). These methods, the single best model, and the ensemble comprising all models are evaluated and compared across multiple benchmark datasets.

The empirical evaluation across the six OpenML datasets showed that Ensemble Selection clearly outperformed both the single best model and the ensemble of all models. GES and DHCEP using UWA achieved the highest predictive performance, with GES doing it at a fraction of the computational cost. Clustering-based selection produced the most diverse and runtime-efficient ensembles. Overall, GES demonstrated the most favorable balance of accuracy, efficiency, and robustness for automated ensemble size determination.