Masterarbeit MSTR-2021-113

Bibliograph.
Daten
Schäufele, Johannes: Improved RAFT architectures for optical flow estimation.
Universität Stuttgart, Fakultät Informatik, Elektrotechnik und Informationstechnik, Masterarbeit Nr. 113 (2021).
91 Seiten, englisch.
Kurzfassung

The estimation of optical flow, that is computing the displacement field between two images, is a useful tool in computer vision that has many applications as part of larger frameworks. RAFT [70], a recent method for optical flow estimation, has significantly improved the quality of results on realistic benchmarks over previous approaches, while simultaneously reducing model complexity and training cost. Despite these advancements, RAFT still has several shortcomings including its flow upsampling that can only capture high-resolution details to a limited extent, simple cost volume without normalization, and limited incorporation of multiple frames in sequences. Due to its novelty, the method has also not been applied to related tasks, such as unsupervised optical flow estimation. To address this, we propose several remedies to these mentioned shortcomings of RAFT, including different cost volume normalization strategies and alternative matching cost functions, as well as different flow upsampling strategies that can capture more high-resolution details. We also extend the method to unsupervised training as well as online training, which involves multiple frames of sequences. In the context of unsupervised training, we introduce learned losses that can be applied to arbitrary model architectures and improve results over traditional photometric and smoothness losses. Our online learning approaches yield an improvement over RAFT’s warm start and use multi-frame consistency to improve performance on video sequences. We evaluate our approaches on optical flow benchmarks and find that our modifications represent improvements over RAFT when working within a limited computational budget. We also argue that these result should scale for training configurations without such limitations.

Volltext und
andere Links
Volltext
Abteilung(en)Universität Stuttgart, Institut für Visualisierung und Interaktive Systeme, Visualisierung und Interaktive Systeme
BetreuerBruhn, Prof. Andrés; Jahedi, Azin; Schmalfuß; Jenny
Eingabedatum21. Mai 2024
   Publ. Institut   Publ. Informatik