Masterarbeit MSTR-2021-72

Bibliograph.
Daten
Rau, Simeon: Visualization for human-AI collaborative music composition.
Universität Stuttgart, Fakultät Informatik, Elektrotechnik und Informationstechnik, Masterarbeit Nr. 72 (2021).
77 Seiten, englisch.
Kurzfassung

We propose an AI-assisted approach based on interactive visualizations to support users in composing music and getting insights into the AI through hyperparameter analysis. Our user-centered approach allows the user to better control the composition by steering the AI’s suggestions. We use symbolic music data and piano rolls as visual music notation for easier understanding for amateur users and interaction with the notes of a melody. As the user requests multiple possible continuation for a given seed melody, and also multiple continuations for each of the previous continuations, a tree or graph structure of melodies occurs. We visualize this structure with an icicle plot, where the nodes are represented by a piano roll, to show the hierarchical structure of the melody samples. To add sorting options for easier sample selection, while still displaying the structure, we added links between the nodes. Both visualizations enable listening to selected melodies. For larger numbers of generated suggestions, we added a similarity-preserving scatterplot to visualize all samples at the same time with different glyphs representing melody samples. The scatterplot improves the efficiency of sample selection, as similar samples are close together and the user can disregard entire neighborhoods if one sample does not fit at all. We support brushing the scatterplot to select neighborhoods for which we then show visual aggregations to allow for insights into groups. To evaluate our design, we conducted a pair analytics study with two participants with limited musical knowledge. Both participants were able to quickly create compositions they liked and found our approach helpful. They also learned new things about the AI, like the influence of the hyperparameter temperature on the resulting melody.

Volltext und
andere Links
Volltext
Abteilung(en)Universität Stuttgart, Institut für Visualisierung und Interaktive Systeme, Visualisierung und Interaktive Systeme
BetreuerSedlmair, Prof. Michael; Heyen, Frank
Eingabedatum15. Februar 2022
   Publ. Institut   Publ. Informatik