Bachelor Thesis BCLR-2025-111

BibliographyAmer, Aly: Extending a deployment platform for AI models with a gRPC API.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Bachelor Thesis No. 111 (2025).
71 pages, english.
Abstract

The increasing adoption of Machine Learning models has created a critical need for flexible and high-performance model serving solutions. While prior work on the NEXON platform established a proof-of-concept forONNXmodel deployment, its REST-only architecture limited both performance and extensibility for future communication protocols, and created a scalability bottleneck, especially for large models. This thesis addresses these limitations by extending the platform into a multi-protocol architecture, creating a foundation for future communication interfaces. Following a Design Science approach, a high-performance gRPC service was implemented and integrated into the existing platform via an Envoy API gateway. The primary goal was to conduct a comprehensive empirical comparison between the new gRPC interface and the existing REST-based solution. The new architecture was subjected to a comprehensive empirical evaluation, assessing its functional correctness, performance, resource utilization, and robustness. The performance results demonstrate the superiority of gRPC for this inference workload, delivering substantial improvements in latency and throughput. Most critically, scalability testing confirmed that the gRPC service resolves the core bottleneck identified in prior work. The evaluation also confirmed that both protocols are functionally equivalent and that system resilience is a function of architecture, not protocol choice. Finally, this work successfully validates gRPC as a high-performance alternative to REST, creating an extensible, multi-protocol foundation for the NEXON platform. It delivers a practical architectural blueprint for integrating alternative communication protocols, transforming the platform from a monolithic proof-of-concept into a flexible, scalable system. Future work could include adding GPU support, evaluating streaming inference, and integrating with container orchestration platforms like Kubernetes.

Department(s)University of Stuttgart, Institute of Software Technology, Empirical Software Engineering
Superviser(s)Wagner, Prof. Stefan; Haug, Markus
Entry dateApril 29, 2026
   Publ. Institute   Publ. Computer Science