Masterarbeit MSTR-2023-36

Bibliograph.
Daten
Walter, Peter: Filter dictionaries for optical flow prediction with RAFT.
Universität Stuttgart, Fakultät Informatik, Elektrotechnik und Informationstechnik, Masterarbeit Nr. 36 (2023).
74 Seiten, englisch.
Kurzfassung

In the field of optical flow estimation, a dense vector field must be generated describing the apparent two-dimensional displacement of objects in consecutive images of a sequence. Although state of the art predictions are currently produced by deep convolutional neural networks, one major issue is that they are strongly susceptible to adversarial attacks, such as the Perturbation Constrained Flow Attack, which create small, noisy perturbations pursuing maximal change in the optical flow estimate. To improve adversarial robustness, this thesis includes receptive field convolutional layers into the optical flow predicting neural network RAFT. These receptive field layers use filter dictionaries to impose specific (geometric) priors onto convolutional kernels and improve results in image classification and reconstruction tasks. Each kernel in these RFCNNs can be written as a weighted sum over a fixed subset of filters taken from the dictionary. Besides the existing Gaussian derivative and Parseval completed sparse directional dictionaries, a novel PCA dictionary is proposed which consists of the principal components of the previously trained network’s kernels. All types of dictionaries are compared against each other at multiple positions in the network. Results show that receptive fields in individual layers mostly do not affect and in RAFT’s feature encoder even degrade performance, while Parseval completed dictionaries do not benefit the neural network in this context of optical flow. However, filter dictionaries with geometric motivations in RAFT’s update block, namely the Gaussian derivatives and sparse directional FDs, make the network up to 20% more robust against the PCFA in exchange for a worse fit in quality.

Volltext und
andere Links
Volltext
Abteilung(en)Universität Stuttgart, Institut für Visualisierung und Interaktive Systeme, Visualisierung und Interaktive Systeme
BetreuerBruhn, Prof. Andrés; Ertl, Prof. Thomas; Schmalfuss, Jenny
Eingabedatum24. Oktober 2023
   Publ. Informatik