Bibliography | Wemmer, Eileen: Progressions of emotion annotations in conversations and dreams for emotion analysis. University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 81 (2023). 108 pages, english.
|
Abstract | Emotion Analysis describes the field of study concerned with the extraction of explicit emotional content implicitly contained in text. To this end, supervised machine learning approaches are commonly employed which rely on annotated data for training and evaluation. This data dictates the tasks trained models will be able to solve and as a consequence, multiple corpora containing emotion annotations have been gathered in the past. These corpora commonly vary in multiple ways, including their underlying domain, the scope of each annotation, and whether they feature emotion representations in terms of categories or their position in a vector space spanned by interpretable dimensions. However, to the best of our knowledge, none of the previously gathered corpora allow for a fine-grained analysis and prediction of the progression of emotional content over the course of texts. While sequential annotations representing the emotional content of each part a text is comprised of exist, they usually only pertain to the parts they are attached to and do therefore not accurately reflect the current emotions expressed in the overall text at this point. Similarly, contextualized annotations that take into account the previous content of a text exist, yet they are not gathered in sequences and therefore also don’t allow for the analysis of the changes in emotional content. This thesis aims to close that gap by gathering progressional labels, that are both sequential and contextualized by the previous text, through a novel, incremental annotation task. In a crowdsourcing setup, texts are revealed to annotators part-by-part and they are asked to annotate the emotional content in terms of categorical labels and appraisal dimensions up to the current point. This yields a set of sequential labels that represent the development of emotional content up to the part of the text they are assigned to. We gather progression annotations for both dream reports and customer service dialogues and find the novel incremental annotation setup to be suitable for their collection. Analyses of the data show that changing progressions exist in both domains, though they are more varied for dream reports than for customer service dialogues in terms of how often annotations change throughout one instance. We do not uncover any clear tendencies in progressions for either domain, implying a rich variation in changes between instances. Leveraging the gathered data to study the degree to which simple sequential models are able to learn to make use of this progressional information, we show a consistent increase in performance for models trained on intact categorical progressions that is too small to be conclusive. This motivates further research, as experiments with stronger baseline systems could help get clearer insights into the matter.
|