Master Thesis MSTR-2025-68

BibliographyMuzaffari, Feda Hussain: Enhancing Safety through Human Factor Monitoring in Virtual Reality.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 68 (2025).
87 pages, english.
Abstract

Human-Robot Collaboration (HRC) is a foundational element of modern Industry 4.0 paradigms, creating synergistic manufacturing environments that combine robotic precision with human flexibility. However, the close proximity and dynamic nature of these interactions introduce significant challenges to worker safety and performance, necessitating advanced training and monitoring solutions. This thesis addresses these challenges through the design, development, and evaluation of an immersive Virtual Reality (VR) training system for HRC tasks. The system’s core innovation lies in its capacity for real-time human factor monitoring, which enables a closed-loop adaptive feedback mechanism to enhance operator safety and proficiency. The developed system integrates a sophisticated suite of non-invasive sensors within a distributed, real-time data architecture. A full-body motion capture suit provides continuous kinematic data, which is processed using a stream processing engine (Apache Flink) to perform a real-time ergonomic risk assessment. This assessment is informed by biomechanical principles derived from established methodologies to identify high-risk postures of the neck, back, and shoulders. Concurrently, an integrated eye-tracker within the VR headset monitors the practitioner’s visual attention, identifying periods of inattention or distraction from critical task areas. For post-session analysis, surface Electromyography (sEMG) data is collected from key upper-limb muscles. Due to hardware sampling rate limitations precluding frequency-domain analysis, this research evaluates trends in muscle load using robust time-domain features, such as Root Mean Square (RMS), which are effective for differentiating muscle states [VNR14]. Based on the real-time analysis of ergonomic and attentional data, the system provides immediate, multimodal feedback to the practitioner through visual and auditory cues within the VR environment. Crucially, it also generates adaptive commands that dynamically alter the behavior of the simulated collaborating robot—a KUKA KR210—by modulating its speed or pausing its actions to mitigate identified risks. This research details the complete system architecture, from the Unity-based VR simulation and ROS 2 robot control interface to the Kafka and Flink backend for stream processing and data persistence. The primary contribution of this work is the creation and planned evaluation of this integrated, adaptive system, which aims to demonstrate the feasibility and potential benefits of real-time, data-driven feedback for creating safer, more effective, and human-aware HRC training environments. Keywords: Human-Robot Collaboration, Virtual Reality, Ergonomics, RULA, Fatigue Monitoring, Eye Tracking, Motion Capture, EMG, Real-Time Feedback, Apache Flink, Kafka, MQTT, Industrial Safety.

Department(s)University of Stuttgart, Institute of Visualisation and Interactive Systems, Visualisation and Interactive Systems
Superviser(s)Sedlmair, Prof. Michael; Kurzhals, Dr. Kuno; Bances Purizaca, Dr. Nelson Enrique; Pathmanathan, Nelusa; Yang, Xiliu
Entry dateDecember 19, 2025
New Report   New Article   New Monograph   Computer Science