Article in Proceedings INPROC-2003-02

BibliographyRantzau, Ralf: Processing Frequent Itemset Discovery Queries by Division and Set Containment Join Operators.
In: Zaki, Mohammed (ed.); Aggarwal, Charu (ed.): Proceedings of the ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery (DMKD), San Diego, California, USA, June 13, 2003.
University of Stuttgart, Faculty of Computer Science.
Report No. 03-05, pp. 20-27, english.
Rensselaer Polytechnic Institute, Troy, New York 12180-3590, USA, June 2003.
Article in Proceedings (Conference Paper).
CR-SchemaH.2.4 (Database Management Systems)
H.2.8 (Database Applications)
Keywordsassociation rule discovery; relational division; set containment join
Abstract

SQL-based data mining algorithms are rarely used in practice today. Most performance experiments have shown that SQL-based approaches are inferior to main-memory algorithms. Nevertheless, database vendors try to integrate analysis functionalities to some extent into their query execution and optimization components in order to narrow the gap between data and processing. Such a database support is particularly important when data mining applications need to analyze very large datasets or when they need access current data, not a possibly outdated copy of it.

We investigate approaches based on SQL for the problem of finding frequent itemsets in a transaction table, including an algorithm that we recently proposed, called Quiver, which employs universal and existential quantifications. This approach employs a table schema for itemsets that is similar to the commonly used vertical layout for transactions: each item of an itemset is stored in a separate row. We argue that expressing the frequent itemset discovery problem using quantifications offers interesting opportunities to process such queries using set containment join or set containment division operators, which are not yet available in commercial database systems. Initial performance experiments reveal that Quiver cannot be processed efficiently by commercial DBMS. However, our experiments with query execution plans that use operators realizing set containment tests suggest that an efficient processing of Quiver is possible.

Full text and
other links
PDF (202152 Bytes)
DMKD 2003
CopyrightAssociation for Computing Machinery (ACM)
Contactrrantzau@acm.org
Department(s)University of Stuttgart, Institute of Parallel and Distributed High-Performance Systems, Applications of Parallel and Distributed Systems
Project(s)ORBIT
Entry dateMay 19, 2003
   Publ. Department   Publ. Institute   Publ. Computer Science