Bachelor Thesis BCLR-2023-27

BibliographyWeiß, Anna-Lena: Deep Learning-based Quality Assurance of Acceptance Criteria for Software Engineers.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Bachelor Thesis No. 27 (2023).
53 pages, english.
Abstract

In the last years, user stories (USs) have gained high popularity in agile software development and there is a lot of research to improve their quality. Although acceptance criteria (AC) are inherently connected to USs, there is a lack of research on their verification and quality assurance. Since they are written in natural language, the typical problems of Natural Language Processing (NLP) also apply to AC. One of the major problems of NLP is ambiguity which involves the lexical interpretation of words and phrases based on context. This thesis proposes the deep learning-based model ACWeakBERT uncased for detecting weak words in English AC that indicate ambiguity along with a set of quality criteria for AC. Based on a traditional narrative literature review, a set of quality criteria for AC is created and their verifiability with rule-based NLP techniques is analyzed. In addition, the most used NLP techniques and existing methods for verifying AC are listed. A dataset is created based on the provided AC and weak words from Bosch Digital (BD). Subsequently, the distilBERT base model is fine-tuned on the preprocessed dataset resulting in ACWeakBERT uncased. The model is evaluated on an unseen test set and achieves 99.55% recall, 98.80% precision, and 99.18% F1-score. In terms of the importance of recall over precision, the model performs sufficiently well in detecting weak words in AC. Compared to the classic rule-based NLP approach of the Automatic Quality User Story Artisan (AQUSA), ACWeakBERT uncased even achieved an improved performance with 26.60% higher precision, 5.75% higher recall, and 17.59% higher F1-score. This thesis is an important step towards the verification and quality assurance of AC by promoting the integration of deep learning models to improve the performance and capabilities of NLP. At the same time, it highlights the importance of further research on the quality assurance of AC and provides ideas for possible future directions.

Department(s)University of Stuttgart, Institute of Software Technology, Empirical Software Engineering
Superviser(s)Wagner, Prof. Stefan; Zimmer, Dr. Ilona; Habib, Mohammad Kasra
Entry dateSeptember 15, 2023
New Report   New Article   New Monograph   Computer Science