| Kurzfassung | Providing cross-device service has become the best practice for many organizations. Those crossdevice services often have different user interfaces (UIs) designed to deliver user experiences (UXs) adapted to the device context. User behavior tends to vary across devices due to the UI variations, which can considerably impact usability. While extensive research exists on the relationship between user behavior and usability within the same device context, little attention has been paid to cross-device scenarios. By examining usage across devices, it becomes possible to identify the distinct user behavioral patterns and assess usability across varying UIs. Usability can be measured explicitly through user responses to a usability questionnaire and implicitly via navigation behavior events and task completion time on web pages. To facilitate this analysis, we developed a recording system capable of capturing user navigation behavior, usability feedback, and screenshots for five popular individual web pages. Participants were recruited both online and onsite to generate a diverse user behavior dataset. A total of 79 participants were invited to the experiment, with 17 participating in a pilot study. The resulting dataset comprises 34 well-processed, complete user datasets for analysis. Using this novel dataset, we conducted a preliminary analysis to explore correlations between user behavior features and usability metrics, specifically System Usability Scale (SUS) scores and task completion times. Additionally, we performed a small case study to compare the five different UIs and determine which design provided the best user experience. We found that usability, measured in terms of SUS scores, showed no significant differences across devices or web pages. However, completion time varied significantly across devices. Additionally, we provided design recommendations, such as addressing erratic user behaviors derived from speed-related metrics like excessive acceleration, which negatively impact usability, particularly on mobile devices.
|