Bachelor Thesis BCLR-2020-68

BibliographyDamyanova, Beloslava: Quality attributes in AI-ML-based systems: differences and challenges.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Bachelor Thesis No. 68 (2020).
84 pages, english.
Abstract

Context: There is an increasing need for ubiquitous distribution and deployment of AI-ML-based systems in industry and public sectors. This is driven by advances in machine learning techniques, including deep learning and neural networks. However, there are specifc problems with machine learning applications in terms of their quality, which reduce trust in these systems. This is especially problematic for systems which fnd application in safety-critical domains like self-driving cars and disease diagnosis. Machine learning-enabled systems learn from data for decision-making and are not designed to meet conventional requirements specifcations. Whether existing standards and principles of quality attributes demand adaptation to the new context is an open research question. Objectives: In this bachelor thesis, we have four objectives. First, we aim to identify if any non-functional requirements from ISO 25010 undergo a shift in ML-enabled systems. Then, we examine whether any new quality characteristics, e.g., trainability, generalizability, fairness, have emerged and need to be added to the standard. Lastly, we intend to determine the most critical and most challenging attributes in AI-ML-based systems. Concerning the objectives mentioned above, it is especially diffcult to receive an overview of the perspective of software practitioners. Some knowledge does exist on the topic, however, it is insuffcient. Our primary goal is, therefore, to fnd and understand the state-of-practice on quality attributes in AI-ML-enabled systems based on experts’ opinions and needs. Method: We conducted a grey literature review to accomplish the goals of our study. We use two different search engines and a QA website to identify literature on quality attributes coming from software practitioners. In total, 91 grey literature sources were selected from which we extracted the detailed knowledge necessary for our research. Results: The results of our research show that for software systems with machine learning components, some modifcations and adjustments of the conventional quality attributes have to be undertaken. We also encountered several unique non–functional requirements for machine learning-enabled systems such as explainability, fairness, trainability, and generalizability. Moreover, we identifed 13 quality attributes as important and challenging to assure, based on the perspective of authoritative software practitioners. We propose that the quality of systems with machine learning components should be monitored and improved based on the quality attributes resulted from our study. Conclusion: To support machine learning practitioners with resolving the challenges associated with AI-ML-based systems, we present an analysis on which quality characteristics should be accommodated for the unique nature of these applications. The limited 3research on quality attributes for machine learning makes our study more needed in the industry at the moment. We believe it provides major opportunities for future research, which results would foster the improvement of AI-ML-based systems.

Full text and
other links
Volltext
Department(s)University of Stuttgart, Institute of Software Technology, Empirical Software Engineering
Superviser(s)Wagner, Prof. Stefan; Bogner, Dr. Justus
Entry dateMarch 1, 2021
   Publ. Computer Science