Master Thesis MSTR-2022-91

BibliographyBihlmaier, Simon Tobias: Optical flow estimation with separable cost volume.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 91 (2022).
75 pages, english.

Optical Flow Estimation is an important task in computer vision that involves finding correspondences between subsequent frames. Recently many approaches have focused on learning to estimate optical flow using neural networks. Constructing and processing correlation volumes using on volutional neural networks is applied in many works and yields good results. Separable Flow by Zhang et al. is an extension for correlation volume based methods such as Recurrent All-Pairs Field Transforms for Optical Flow by Teed and Deng. It separates the four dimensional correlation volume into two correlation volumes with only one instead of two displacement dimensions. At the time of its release, state of the art estimation quality results were reported for Separable Flow on the Sintel and KITTI datasets. By investigating the implementation provided by the authors, significant changes of the model structure and training schedule compared to the paper can be discovered. The goal of this thesis is to verify the published claims about the training regime, model structure, estimation quality and number of parameters. This is accomplished by reverting identifiable changes in multiple ablation steps. Evaluating the ablation step closest to the published description shows that the claimed estimation quality can not be reproduced. The provided model implementation combined with the published training schedule performs the most similar to the results of the paper. Additionally, the claim that the four dimensional correlation volume does not need to be stored in order to compute the three dimensional correlation volumes is investigated.This claim is verified by providing an alternative parallel implementation for Graphics Processing Units that fulfills the storage constraint. At the cost of longer computation times, the memory consumption of Separable Flow can be reduced during training and inference. In an effort to improve the estimation quality, Global Motion Aggregation by Jiang et al. is added to Separable Flow. On the ablation training schedule, the combined model achieves better results than Global Motion Aggregation in isolation.

Full text and
other links
Department(s)University of Stuttgart, Institute of Visualisation and Interactive Systems, Visualisation and Interactive Systems
Superviser(s)Bruhn, Prof. Andres; Jahedi, Azin
Entry dateApril 17, 2023
   Publ. Computer Science