Master Thesis MSTR-2024-143

BibliographyBauer, Katrin: Effective robustness for optical flow.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 143 (2024).
69 pages, english.
Abstract

Understanding the motion is crucial for many computer vision tasks. The optical flow is a formal description of the apparent 2-dimensional motion as a dense pixel matching between two video frames. Its computation is a fundamental building block for many applications including action recognition, video processing and robot navigation. However, computing the optical flow is still challenging. Most recent advances were made by applying deep learning. Neural networks are trained by optimizing millions of parameters to the training data. When the training data is limited, they're prone to overfit and not generalize well to different data distributions. Quantifying robustness is crucial for understanding current limitations and evaluating the effectiveness of robustification strategies. Previous research in the context of image classification suggests that the out-of-distribution and in-distribution accuracy are strongly correlated. The effective robustness metric by Taori et al. quantifies robustness while controlling for this correlation. This thesis transfers the concept of effective robustness from image classification to optical flow. Based 21 different architectures evaluated on 6 different distribution shifts, we find that the in-distribution performance serves as a good predictor for the out-of-distribution performance. Based on this finding we establish a baseline for quantifying robustness and analyze several state-of-the-art models with respect to that baseline. One approach to improve the robustness of neural networks is weightspace averaging. Given several trained checkpoints of the same architecture, weightspace averaging constructs a new checkpoint by averaging the weight matrices. In the context of image classification, averaging the weights of multiple fine-tuned checkpoints has been shown to improve generalization without increasing inference time. We apply weightspace averaging as approach to improve the robustness of the optical flow model RAFT.

Full text and
other links
Volltext
Department(s)University of Stuttgart, Institute of Visualisation and Interactive Systems, Visualisation and Interactive Systems
Superviser(s)Bruhn, Prof. Andrés; Schmalfuss, Jenny
Entry dateMay 18, 2026
   Publ. Institute   Publ. Computer Science