Master Thesis MSTR-2026-43

BibliographyPeys, Daniel: Semantic augmentation of data product metamodels to enable LLM-access through MCP.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 43 (2026).
152 pages, english.
Abstract

Data products are a central concept in data mesh architectures, but their use through LLM-based interfaces requires standardized access mechanisms and semantic context for correct interpretation. Existing data product metamodels do not yet define LLM-based access and semantic models as integrated concepts. This thesis addresses this gap by extending the ProMoTe metamodel with concepts for LLM-based access and semantic models. Furthermore, it compares different semantic model approaches and retrieval strategies with respect to answer quality and efficiency to assess their suitability for LLM-based data product access.

Based on defined system requirements and an analysis of existing metamodels, a system concept and architecture are derived from the extended ProMoTe metamodel. The architecture uses MCP for LLM-based access and is implemented in three variants, namely a non-augmented baseline without a semantic model, a RAG-based variant, and a KG-based variant. The KG-based variant is further divided into two configurations, KG-Only and KG+Re-Rank, with KG+Re-Rank applying re-ranking to the retrieved triples used as semantic context.

All variants are evaluated to validate the extension in practice and to assess them with respect to answer quality and efficiency. To this end, 50 manually created questions covering both single-hop and multi-hop reasoning across different question types were designed. During the evaluation, efficiency metrics are collected automatically, while quality metrics are assessed manually against the underlying semantic model and the data product API. The results show that all semantic-model-based variants substantially improve accuracy compared to the non-augmented baseline, indicating the importance of providing a semantic model. Among the semantic-model-based variants, KG+Re-Rank achieves the best accuracy, while the RAG-based and KG-Only variants achieve the same result. The results and error analysis further indicate that the format of the semantic model, namely structured triples or natural language, does not substantially affect accuracy when the correct semantic context is retrieved. Instead, retrieval precision and question decomposition quality have a stronger impact on answer quality.

Regarding efficiency, the non-augmented variant achieves the best results, but this comes at the cost of substantially lower response quality. For the semantic-model-based variants, the results indicate that the natural language format of the RAG-based semantic model does not reduce efficiency compared to the structured triples used by the KG-based variants. This can be explained by the fact that response time is affected more strongly by completion tokens than by prompt tokens, while the semantic model format mainly influences the number of prompt tokens. Since the additional tokens used by the RAG-based variant are primarily prompt tokens, its higher total token usage does not translate into longer average response times compared to the other semantic-model-based variants.

Full text and
other links
Volltext
Department(s)University of Stuttgart, Institute of Parallel and Distributed Systems, Applications of Parallel and Distributed Systems
Superviser(s)Schwarz, Prof. Holger; Behringer, Dr. Michael; Schuiki, Laura; Ratnamaheson, Niven
Entry dateAugust 13, 2026
   Publ. Institute   Publ. Computer Science