Master Thesis MSTR-2024-86

BibliographyWagner, Frederik: Exploring retrieval-augmented language modeling for material prediction of vehicle components.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 86 (2024).
66 pages, english.
Abstract

Recent advances in natural language processing (NLP), particularly in large language models (LLMs) like ChatGPT, demonstrate the potential for their application in a variety of tasks within specialized domains. For instance, in the automotive domain, they could be utilized to provide guidance through a vehicle repair process. This thesis is concerned with the problem of predicting suitable materials for vehicle components, such as brake discs. It seeks to determine whether LLMs can draw on both world and domain-specific knowledge to make accurate predictions about component materials without requiring extensive fine-tuning. This is achieved through retrieval-augmented generation (RAG), which involves retrieving relevant information from external sources and using it to enhance the prompt. Specifically, this work compares three approaches: a standard LLM model, a simple RAG approach, and an iterative RAG method called Chain-of-Verification (CoVe). The thesis also develops a custom annotation tool to facilitate a human evaluation study due to the absence of a gold standard dataset. Results demonstrate that LLMs perform well in material prediction tasks, and while both RAG approaches do not significantly enhance prediction quality, they do not detract from it either. This research concludes that LLMs, with or without retrieval augmentation, offer a promising solution for material prediction in vehicle components, though challenges in evaluation, hyperparameter optimization, and data retrieval persist.

Full text and
other links
Volltext
Department(s)University of Stuttgart, Institute for Natural Language Processing
Superviser(s)Schulte im Walde, Prof. Sabine; Kuhn, Prof. Jonas; Eichel, Annerose
Entry dateMarch 13, 2025
   Publ. Computer Science