Master Thesis MSTR-2022-24

BibliographyReichel, Maximilian: Metamorphic testing of version control systems.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 24 (2022).
61 pages, english.

Currently, no approach exists to automatically and systematically test the implementation of version control systems (VCS). VCS are widely used in the software development industry, where a lot of people rely on these systems to perform as specified and not to lose any data. However, problems with the implementations of such systems are found frequently which causes risks and inconveniences to the users. The oracle problem renders traditional testing impractical. In this thesis, we present an approach to automatically test the implementation of VCS using metamorphic testing. Metamorphic testing is an approach to mitigate the oracle problem by splitting it up into smaller parts. Instead of knowing the expected behavior of a program for a specific input, this approach uses the knowledge of how a certain change of the program input will be reflected in its output. We develop such transformations of inputs of VCS and the corresponding relations of the outputs. Since this requires initial inputs to work with, we also develop a random input test generator. We combine these two components into an automated testing tool. As inputs, we use bash-scripts containing commands to interact with the VCS and the file system. As outputs, we use the files in the working directory after executing the input scripts. Additionally, we develop a test minimizer to reduce the number of commands in the inputs when a relation gets violated. This eases the manual analysis later. Since git is complex and currently the most popular VCS, it makes a good first target to test our approach. We find five real-world bugs in the implementation of git and archive a precision of 90.79%. Additionally, we are able to minimize the test inputs to 15 of the original size on average.

Full text and
other links
Department(s)University of Stuttgart, Institute of Software Technology, Software Lab - Program Analysis
Superviser(s)Pradel, Prof. Michael; Christakis, Dr. Maria
Entry dateSeptember 16, 2022
   Publ. Computer Science