ABSTRACT

Techniques for efficient and distributed processing of huge, unbound data streams have made some impact in the database community. Sensors and data sources, such as position data of moving objects, continuously produce data that is consumed, e.g., by location-aware applications. Depending on the domain of interest, e.g. visualization, the processing of such data often depends on domain-specific functionality. This functionality is specified in terms of dedicated operators that may require specialized hardware, e.g. GPUs. This creates a strong dependency which a data stream processing system must consider when deploying such operators. Many data stream processing systems have been presented so far. However, these systems assume homogeneous computing nodes, do not consider operator deployment constraints, and are not designed to address domain-specific needs.

In this paper, we identify necessary features that a exible and extensible middleware for distributed stream processing of context data must satisfy. We present NexusDS, our approach to achieve these requirements. In NexusDS, data processing is specified by orchestrating data flow graphs, which are modeled as processing pipelines of predefined and general operators as well as custom-built and domain-specific ones. We focus on easy extensibility and support for domain-specific operators and services that may even utilize specific hardware available on dedicated computing nodes