This bachelor thesis tackles the lack of systematic documentation of codebook changes in text annotation projects by developing a user-friendly Expert Decision UI. This user interface extends the Annotation Tool to support experts in recording their decision-making processes. The user interface was developed through conducting an expert interview and two usability studies, with the objective of obtaining maximum feedback and identifying potential issues in a timely manner. A conducted recall test further indicated that providing additional context increases confidence and ease of understanding a previous codebook change. The developed solution contributes to the advancement of annotation tools by introducing features for tracking codebook changes, which are not yet available in comparable systems.