Classifier ensemble algorithms allow for the creation of combined machine learning models that are more accurate and generalizable than individual classifiers. However, creating such an ensemble is complex, as several requirements must be fulfilled. An expert has to select multiple classifiers that are both accurate and diverse. In addition, a decision fusion algorithm must be selected to combine the predictions of these classifiers into a consensus decision. Satisfying these requirements is challenging even for experts, as it requires a lot of time and knowledge. In this position paper, we propose to automate the creation of classifier ensembles. While there already exist several frameworks that automatically create multiple classifiers, none of them meet all requirements to build optimized ensembles based on these individual classifiers. Hence, we introduce and compare three basic approaches that tackle this challenge. Based on the comparison results, we propose one of the approaches that best meets the requirements to lay the foundation for future work.