Recent advancements in speech recognition and related technologies have led to a significant increase in the use of voice assistant systems. These systems, now integrated into various devices such as mobile phones and cars, are increasingly preferred for interacting with digital systems. In this increasing industry, it is very important to evaluate speech systems in a standardized manner. To do this, however, several conditions have to be set, which is challenging for traditional subjective evaluation methods with human usability tests. The goal of this thesis is to develop an automated objective evaluation method for the assessment of voice assistant systems inside vehicle, which requires minimal human supervision. The proposed method seeks to surpass human assessment in benchmarking accuracy and reliability. To achieve this goal, a comprehensive literature review was conducted to understand the functionality of voice assistant systems and to define both subjective and objective evaluation methods. The review highlighted the advantages and disadvantages of each approach. The thesis then details the methodology of the proposed objective evaluation method and presents the benchmark results for each evaluated vehicle. Additionally, a survey involving ten experts from the infotainment field was conducted to assess the usability and effectiveness of the evaluation method. Combining the results of the survey, the benchmark scores, and the information gathered threw the literature review, reveals that evaluating with subjective human methods lack important characteristics when comparing the evaluated test subjects. In contrast, objective evaluation methods capture these essential features, making them a superior choice for benchmarking voice assistant systems.