| Bibliography | Le Dac, Huy: Semi-automated qualitative software architecture risk analysis with ATAM using retrieval-augmented generation. University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Bachelor Thesis No. 37 (2025). 143 pages, english.
|
| Abstract | Context. The Architecture Tradeoff Analysis Method (ATAM) is an established approach for systematically evaluating software architectures, focusing on the identification of trade-offs, risks, and sensitivity points based on quality attribute scenarios. It has become a core method in the domain of qualitative software architecture analysis. Problem. Despite its effectiveness, the traditional ATAM process is highly manual and resourceintensive. It requires considerable effort from stakeholders, domain experts, and architects, which often makes the process time-consuming and inefficient, particularly in early-stage architectural evaluations. Objective. With the emergence of Artificial Intelligence and Large Language Models (LLMs), new opportunities arise for automating knowledge-intensive processes such as ATAM. This thesis proposes a semi-automated qualitative analysis tool based on Retrieval-Augmented Generation (RAG) to assist in identifying architectural trade-offs, sensitivity points, and risks from structured inputs. Method. We began with a literature review of existing approaches for (semi-)automated software architecture evaluation. Subsequently, a prototype combining ATAM and RAG was developed, capable of processing structured inputs and retrieving relevant architectural knowledge from document corpora. Prompt engineering was applied to guide the model’s responses. The prototype was evaluated through comparative experiments (manual vs. automated analysis) and a user study. Result. The tool with its hardware configurations reduced analysis time by approximately 90% compared to the manual approach. Many generated risks, trade-offs, and sensitivity points were deemed relevant by experts. However, the tool struggled with deeply contextual insights, occasionally producing redundant or generic content. The user study revealed strengths in responsiveness and usability but also identified challenges in output content, especially lacking in depth. Conclusion. The RAG-enhanced ATAM prototype demonstrates potential to augment architectural analysis by improving efficiency and partially automating risk identification. Nevertheless, it does not yet replace expert-driven judgment. Limitations include dependency on prompt quality, and occasional lack of depth in the results. Future work should focus on enhancing prompt design, integrating visual model interpretation, and improving guidance and explanation features to make the tool more robust and widely applicable.
|