| Bibliography | Beck, Julius: Requirements Engineering with Generative AI: RAG for Assessing Requirements Quality. University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Bachelor Thesis No. 96 (2025). 93 pages, english.
|
| Abstract | Context. This thesis investigates the application of Generative AI (GenAI), particularly Large Language Models (LLMs), for automated quality assessment of software and system requirements. Within the broader field of software engineering, the work focuses on requirements engineering, an area increasingly influenced by AI yet still underexplored in academic literature. Problem. Current AI-based approaches to requirements quality assurance are limited in scope and often rely on classification-based models such as BERT or similar architectures. These models typically output predefined labels without offering detailed explanations, making them insufficient for practical use in contexts where interpretability and reasoning are crucial. Furthermore, existing solutions lack support for collaborative interactions between AI and human experts and are tested mostly on small datasets. Objective. The goal of this thesis is to develop and evaluate a framework that integrates LLMs with retrieval-augmented generation (RAG) to support semi-automated, explainable quality checks of software and system requirements. In addition to the technical implementation, the objective is to empirically assess the extent to which such a system can enhance the effectiveness of human requirement reviews, specifically by improving consistency, identifying subtle defects more reliably, and providing interpretable justifications for detected issues. Method. A demonstrator tool was implemented, combining LLMs, RAG techniques, and a user interface to facilitate human-in-the-loop quality assessments. The system was evaluated through a study comparing its performance to human experts using an industry dataset of smart home requirements, which was enhanced with typical defect patterns. Evaluation metrics included precision, recall, and inter-rater agreement via Fleiss’ Kappa. Additionally, an expert interview was conducted to assess the tools usefulness. Result. The LLM-based system achieved high recall in detecting quality issues, outperforming human experts in identifying a broader range of potential errors. However, it also exhibited a high false positive rate. While human experts demonstrated higher precision, they often overlooked subtle inconsistencies. The findings indicate that the combination of LLMs and human reviewers is likely to be more effective than using either in isolation. Conclusion. This work presents a robust proof of concept for the integration of LLM-based systems into requirement quality assurance workflows. The developed approach demonstrably supports human reviewers and contributes to more comprehensive and consistent assessments. While limitations such as hallucinations and overgeneralizations persist, the results underline the potential of explainable AI systems to augment expert judgment in requirements engineering and provide a foundation for further research and practical refinement.
|