Article in Proceedings INPROC-2020-55

BibliographyGiebler, Corinna; Gröger, Christoph; Hoos, Eva; Schwarz, Holger; Mitschang, Bernhard: A Zone Reference Model for Enterprise-Grade Data Lake Management.
In: Proceedings of the 24th IEEE Enterprise Computing Conference.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology.
pp. 57-66, english.
IEEE, October 2020.
Article in Proceedings (Conference Paper).
CR-SchemaH.4 (Information Systems Applications)
KeywordsData Lake; Zones; Reference Model; Industry Case; Industry Experience
Abstract

Data lakes are on the rise as data platforms for any kind of analytics, from data exploration to machine learning. They achieve the required flexibility by storing heterogeneous data in their raw format, and by avoiding the need for pre-defined use cases. However, storing only raw data is inefficient, as for many applications, the same data processing has to be applied repeatedly. To foster the reuse of processing steps, literature proposes to store data in different degrees of processing in addition to their raw format. To this end, data lakes are typically structured in zones. There exists various zone models, but they are varied, vague, and no assessments are given. It is unclear which of these zone models is applicable in a practical data lake implementation in enterprises. In this work, we assess existing zone models using requirements derived from multiple representative data analytics use cases of a real-world industry case. We identify the shortcomings of existing work and develop a zone reference model for enterprise-grade data lake management in a detailed manner. We assess the reference model's applicability through a prototypical implementation for a real-world enterprise data lake use case. This assessment shows that the zone reference model meets the requirements relevant in practice and is ready for industry use.

ContactSenden Sie eine E-Mail an corinna.giebler@ipvs.uni-stuttgart.de
Department(s)University of Stuttgart, Institute of Parallel and Distributed Systems, Applications of Parallel and Distributed Systems
Entry dateDecember 7, 2020
   Publ. Department   Publ. Institute   Publ. Computer Science