Bachelor Thesis BCLR-2020-13

BibliographyZahariev, Denis: Controlled User Study: Usability and Efficiency Evaluation of the Parallel Performance Catalogue Extension for the Palladio-Bench.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Bachelor Thesis No. 13 (2020).
61 pages, english.
Abstract

For the last two decades, the number of cores in modern CPUs has been steadily increasing. This enables a significant leap in the performance of modern software when the right parallel programming approaches and strategies are being used.

One tool used by software performance engineers to examine and evaluate the performance and reliability of pieces of software is the Palladio-Bench. This tool allows its users to analyse these and many more Quality of Software (QoS) properties such as sizing, scalability, and load balancing, based only on graphical models of the software architecture. After various pieces of research showed that the Palladio-Bench does not fully support parallelism, and the modelling of parallel programming strategies, a new extension for the tool was developed. This new extension for the Palladio-Bench incorporates fundamental parallel programming approaches into its already existing toolkit. The researchers that proposed the extension also claimed that it has higher usability and better time efficiency than the standard modelling toolkit of the Palladio-Bench. However, they were not able to prove it since the extension was not yet developed.

The purpose of this thesis is to put the supposed usability gains to the test. It compares the standard toolkit and the new extension in the context of the modelling of parallel behaviours. To support this study, a set of research questions was defined. The chosen research method was the conduction of a controlled empirical user study. Sixteen participants were recruited and split into two groups. Each group had to complete different modelling tasks with the standard toolkit and the extension. While they were working on the tasks, several metrics were recorded: task completion time, time spent in errors, number of errors, and usability evaluation. Afterwards, this data was statistically analysed and tested.

The results of the analysis prove that the extension increases the usability and the time efficiency of the Palladio-Bench. A reduction in the time spent in errors and the number of errors, however, could not be proved.

Department(s)University of Stuttgart, Institute of Software Technology, Software Reliability and Security
Superviser(s)Becker, Prof. Steffen; Frank, Markus
Entry dateJuly 20, 2020
   Publ. Computer Science