Bachelor Thesis BCLR-2024-01

BibliographyHollenbeck, Jo: GPT-4-based visualization reasoning dataset.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Bachelor Thesis No. 1 (2024).
49 pages, english.

Visual data, such as charts and tables, is a widely used method to summarise data clearly. With the ascent of Artificial Intelligence lots of models have been created that provide users with answers to their questions on visual data. An analysis of the latest research reveals that the model accuracy and the reasoning of the results are still insufficient. These two major aspects are tackled in this work. The AI model used is OpenAI’s large language model GPT-4. Tables are presented as text-only input, and charts are uploaded as images to GPT-4. A modified prompt guarantees a step-by-step reasoning as output. With the collected data, quantitative analyses are conducted to evaluate the numerical data and its influence on the response. Moreover, a qualitative analysis is performed determining the quality of answer in terms of clarity, relevance and reasoning. Additionally, responses on tables and charts are compared to get deeper insights on the model’s performance. Notable results are GPT-4’s outstanding performance on the accuracy of the input formats, except for line charts and charts containing dense information. The model consistently produces good-quality answers when provided with either text-only or image-text input. This work demonstrates that GPT-4 performs well on visual data methods, but especially for complex chart images it exhibits room for improvement.

Full text and
other links
Department(s)University of Stuttgart, Institute of Visualisation and Interactive Systems, Visualisation and Interactive Systems
Superviser(s)Bulling, Prof. Andreas; Wang, Yao
Entry dateApril 5, 2024
   Publ. Computer Science