Article in Proceedings INPROC-2024-10

BibliographyTreder-Tschechlov, Dennis; Fritz, Manuel; Schwarz, Holger; Mitschang, Bernhard: Ensemble Clustering based on Meta-Learning and Hyperparameter Optimization.
In: Proceedings of the VLDB Endowment (ed.): Proc. VLDB Endow. 17, 11 (July 2024), 2880–2892..
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology.
pp. 2880-2892, english.
ACM, August 2024.
DOI: https://doi.org/10.14778/3681954.3681970.
Article in Proceedings (Conference Paper).
CR-SchemaI.5.3 (Pattern Recognition Clustering)
Abstract

Efficient clustering algorithms, such as k-Means, are often used in practice because they scale well for large datasets. However, they are only able to detect simple data characteristics. Ensemble clustering can overcome this limitation by combining multiple results of efficient algorithms. However, analysts face several challenges when applying ensemble clustering, i. e., analysts struggle to (a) efficiently generate an ensemble and (b) combine the ensemble using a suitable consensus function with a corresponding hyperparameter setting. In this paper, we propose EffEns, an efficient ensemble clustering approach to address these challenges. Our approach relies on meta-learning to learn about dataset characteristics and the correlation between generated base clusterings and the performance of consensus functions. We apply the learned knowledge to generate appropriate ensembles and select a suitable consensus function to combine their results. Further, we use a state-of-the-art optimization technique to tune the hyperparameters of the selected consensus function. Our comprehensive evaluation on synthetic and real-world datasets demonstrates that EffEns significantly outperforms state-of-the-art approaches w.r.t. accuracy and runtime.

Department(s)University of Stuttgart, Institute of Parallel and Distributed Systems, Applications of Parallel and Distributed Systems
Project(s)VALID-PARTITION
Entry dateSeptember 10, 2024
   Publ. Department   Publ. Institute   Publ. Computer Science