| Kurzfassung | The conventional approach of filter-based search of academic papers is often linear and user-intensive. Initially, it involves manually classifying the relevant academic papers stored in a repository into filter categories. Subsequently, based on the user’s selection, relevant academic papers are suggested. Such an approach is followed in phase two of the framework for migrations to microservices, implemented in the Architecture Refactoring Helper (ARH) tool, that is currently being developed within the ESE group of the Institute of Software Engineering. This tool provides software architects or developers with the task of architectural refactoring of monolithic applications to microservices, a structured guide on selecting an appropriate strategy and refactoring approach. In this master thesis, we propose a more efficient alternative by leveraging the power of Large Language Models (LLMs), a type of artificial intelligence (AI) program that allows for recognizing and generating text in a human-like manner. We use Retrieval-Augmented Generation (RAG) which enhances search capabilities by integrating semantic search strategies to retrieve data and LLMs to generate text. We first conduct an exhaustive review of Retrieval Augmented Generation (RAG) and each of the components in RAG. We then extract the requirements for the development of such a tool and map them to technical value based on our review. In this paper, we present our achievements in the implementation of various tool components, including document retrieval and search methods, text generation, and initial performance evaluation. First, we explore techniques to develop a pipeline that comprises two components: PDF ingestion and context-aware generation. Having developed the pipeline in the first step, it further requires a user interface like Chat-GPT for users. We design this interface in the second step, utilizing the best-suited framework for development. Thus, by integrating the components built in the two steps, we create a tool that generates relevant responses with sources for user queries in natural language. Finally, we experimented with different LLMs, evaluating each for relevant responses, and found our best performing LLM among the selected LLMs. In addition, we evaluate the relevance, coherence of the responses, and usability of the tool through a survey. The nine participants in the survey were software architects or developers with experience in refactoring monoliths into microservices. Overall, the usability feedback from the survey was positive, with some suggestions for improvement. Additionally, the responses from the tool were rated relevant by most users, except for one. This thesis systematically designed, implemented, and evaluated a tool as an enhancement to the ARH tool, providing context-aware paper suggestions using natural language. In addition, the tool enhances existing research by incorporating chat history and follow-up question capabilities. It also enhances user interaction and context. Unlike simple lexical matching, our tool uses semantic similarity for more efficient retrieval.
|