Article in Proceedings INPROC-2011-84

BibliographyRizou, Stamatia; Dürr, Frank; Rothermel, Kurt: Fulfilling End-to-End Latency Constraints in Large-scale Streaming Environments.
In: Proceedings of the 30th IEEE International Performance Computing and Communications Conference: IPCCC'11.
University of Stuttgart : Collaborative Research Center SFB 627 (Nexus: World Models for Mobile Context-Based Systems).
pp. 1-8, english.
IEEE Xplore, November 2011.
Article in Proceedings (Conference Paper).
CR-SchemaC.2.4 (Distributed Systems)
C.2.2 (Network Protocols)
C.2.3 (Network Operations)
Abstract

The on-line processing of high volume data streams is a prerequisite for many modern applications relying on real-time data such as global sensor networks or multimedia streaming. In order to achieve efficient data processing and scalability w.r.t. the number of distributed data sources and applications, in-network processing of data streams in an overlay network of data processing operators has been proposed. For such stream processing overlay networks, the placement of operators onto physical hosts plays an important role for the resulting quality of service—in particular, the endto- end latency—and network load. To this end, we present an enhanced placement algorithm that minimizes the network load put onto the system by a stream processing task under userdefined delay constraints in this paper. Our algorithm finds first the optimal solution in terms of network load and then degrades this solution to find a constrained optimum. In order to reduce the overhead of the placement algorithm, we included mechanisms to reduce the search space in terms of hosts that are considered during operator placement. Our evaluations show that this approach leads to an operator placement of high quality solution while inducing communication overhead proportional only to a small percentage of the total hosts.

Full text and
other links
PDF (136980 Bytes)
CopyrightThis material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE (contact pubs-permissions@ieee.org). By choosing to view this document, you agree to all provisions of the copyright laws protecting it.
Department(s)University of Stuttgart, Institute of Parallel and Distributed Systems, Distributed Systems
Project(s)SFB-627, E3 (University of Stuttgart, Institute of Parallel and Distributed Systems, Distributed Systems)
Entry dateDecember 12, 2011
   Publ. Department   Publ. Institute   Publ. Computer Science