Master Thesis MSTR-2022-120

BibliographyMechraoui, Abdelrahman Lamine: Photorealistic Eye Image Generation through Image-to-Image Translation.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 120 (2022).
64 pages, english.

Appearance-based gaze estimation methods have gained substantial interest due to their easy setup. The only requirement is an off-the-shelf RGB camera and a machine leaning model that regress gaze direction from eye images. However, machine learning models are highly dependent on high quality large-scale datasets. The collection process of such datasets is labour intensive, time consuming, costly, and prone to errors. We propose a generative pipeline that utilizes an established redirection network, and a style-based generative adversarial network. In order to generate large synthetic gaze datasets that vary in terms of eye shape, color, and direction, we employ semantic region-adaptive normalization (SEAN). An established approach that allows finegrained control over the generation process. SEAN uses image generation to control the styles of generated photorealistic images in a semantically meaningful way. SEAN uses image-to-image translation approach where style is deduced by means of segmentation masks’ labels. The resulting dataset realism and usability is evaluated on a series of gaze estimation experiments. The experiments attempt to answer questions related to style, such as whether style is relevant for gaze estimation, and whether style diversity affects gaze estimators. In addition, we study the efficacy of gaze rediection network in augmenting gaze datasets. The resulting dataset was used to augment the popular xGaze dataset in terms of style and gaze directions. Our experiments show that style augmentation does indeed boost the gaze estimator’s performance, it reduces the gaze angular error (4.86◦ →4.45◦); however, gaze redirection experiments fail to add any useful knowledge to the existing gaze dataset.

Department(s)University of Stuttgart, Institute of Visualisation and Interactive Systems, Visualisation and Interactive Systems
Superviser(s)Bulling, Prof. Andreas; Bace, Dr. Mihai
Entry dateApril 8, 2024
   Publ. Computer Science