Article in Proceedings INPROC-2019-26

BibliographySlo, Ahmad; Bhowmik, Sukanya; Flaig, Albert; Rothermel, Kurt: pSPICE: Partial Match Shedding for Complex Event Processing.
In: Proceedings of the 2019 IEEE International Conference on Big Data (BigData '19); Los Angeles, CA, USA 9 - 12 December, 2019.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology.
pp. 1-11, english.
IEEE, December 9, 2019.
Article in Proceedings (Conference Paper).
CR-SchemaC.2.4 (Distributed Systems)
Abstract

Complex event processing (CEP) systems continuously process input event streams to detect patterns. Over time, the input event rate might fluctuate and overshoot the system’s capabilities. One way to reduce the overload on the system is to use load shedding. In this paper, we propose a load shedding strategy for CEP systems which drops a portion of the CEP operator’s internal state (a.k.a. partial matches) to maintain a given latency bound. The crucial question here is how many and which partial matches to drop so that a given latency bound is maintained while minimizing the degradation in the quality of results. In the stream processing domain, different load shedding strategies have been proposed that mainly depend on the importance of individual tuples. However, as CEP systems perform pattern detection, the importance of events is also influenced by other events in the stream. Our load shedding strategy uses Markov chain and Markov reward process to predict the utility/importance of partial matches to determine the ones to be dropped. In addition, we represent the utility in a way that minimizes the overhead of load shedding. Furthermore, we provide algorithms to decide when to start dropping partial matches and how many partial matches to drop. By extensively evaluating our approach on three real-world datasets and several representative queries, we show that the adverse impact of our load shedding strategy on the quality of results is considerably less than the impact of state-of-the-art load shedding strategies.

Full text and
other links
PDF (460107 Bytes)
Department(s)University of Stuttgart, Institute of Parallel and Distributed Systems, Distributed Systems
Entry dateNovember 12, 2019
   Publ. Department   Publ. Institute   Publ. Computer Science