Diploma Thesis DIP-2139

BibliographyWetzel, Matthias: Document-Driven Black-Box-Testing.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Diploma Thesis No. 2139 (2004).
132 pages, english.
CR-SchemaD.2.1 (Software Engineering Requirements/Specifications)
D.2.4 (Software Engineering Software/Program Verification)
D.2.5 (Software Engineering Testing and Debugging)
D.2.9 (Software Engineering Management)
D.2.11 (Software Engineering Software Architectures)
K.6.3 (Software Management)
K.6.4 (System Management)
KeywordsFormal specification; formal documentation; documentation of embedded systems; automatic test; specification coverage; automatic evaluation of test results
Abstract

Software running on embedded systems is playing an increasingly important role in safetycritical and mission-critical applications. Formal documentation methods, like the Tabular Expressions invented by Parnas et al., avoid the ambiguities inherent in natural language by using mathematical notation. This allows the intended behavior of software to be precisely defined, reducing the number of errors that are caused by misleading documentation. Despite its shortcomings, software testing is considered an important part of software quality assurance. A fundamental assumption of software testing is the existence of some mechanism, which is called an oracle, that is able to determine whether the results of a test execution are correct. The oracle is often implemented by manually deriving the expected output from informal and possibly inconsistent documentation and then comparing it to the actual test results. This procedure is both time-consuming and error-prone. However, if a formal specification exists, it can be used to automatically evaluate the test results and determine the success or failure of a test execution. This thesis describes the design and implementation of a working prototype, which combines the use of Tabular Expressions and embedded-systems products developed by Ashling Microsystems Ltd. in a tool that is able to automatically execute function tests on a target embedded system, and evaluate their results using an existing formal specification of the tested function. In addition to testresult-evaluation, the tool is able to determine the specification-coverage of a given set of testcases on the basis of a formal specification, without depending on knowledge or even existence of program code. This allows measuring and improving test quality at any phase of a software project, even prior to implementation. A small trial application demonstrates the usefulness of the methods described in this work. Possible applications in projects and expected benefits are discussed. It is shown that using tests based on formal documentation enforces consistency between documentation and implementation, which greatly increases the value of such a documentation, especially to maintenance programmers. It is further argued that tools performing such tests help to reduce costs for software testing while improving test quality, and by doing so more than compensate the additional costs of creating formal specifications.

Full text and
other links
PDF (1232762 Bytes)
Access to students' publications restricted to the faculty due to current privacy regulations
ContactM. Wetzel wetzel@informatik.uni-stuttgart.de
Department(s)University of Stuttgart, Institute of Software Technology, Software Engineering
Entry dateMarch 18, 2004
   Publ. Computer Science