Master Thesis MSTR-2018-79

BibliographyJahedi, Azin: Improved descriptor Learning for correspondence problems.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology, Master Thesis No. 79 (2018).
113 pages, english.

Solving correspondence problems is a fundamental task in computer vision. In the past decades, many approaches tried to find the matches between images. One way to solve this task is to use feature-based methods, such as SIFT, SURF and DAISY. The mentioned methods are based on engineered features. Learned features are another type of descriptors that are basically learned and computed via a convolutional neural network (CNN). This type of descriptors has gained a lot of attention in the past years. In this thesis, we design and train many CNN to find an architecture and a set of parameters, so that it computes suitable descriptors for the optical flow estimation task. We implement a fast way of computing the descriptors from images, based on a strategy suggested by Bailer et al. After computing the non-dense set of matches by the Coarse-to-Fine PatchMatch (CPM) algorithm, we use the interpolation technique which is introduced and used in Edge-Preserving Interpolation of Correspondences for Optical Flow (Epicflow), to compute the dense flow field. We use the approach of CPM, which is a popular method to estimate optical flow for large displacements. We embed our trained CNN model into CPM in such a way that the algorithm uses the learned descriptors to find the matches in a coarse-to-fine manner. We use the recent benchmarks of KITTI 2015 and MPI-Sintel to train and evaluate our CNN. To this end, we compare the estimated optical flow to the ground truth optical flow provided in the mentioned benchmarks. By computing the Average End-Point Error (AEE) of the obtained optical flow, we thus have a measure to assess the learned descriptors and we can use it as a feedback to change the network so that it leads to more suitable descriptors. In this way, we were able to design and train a network that computes descriptors which perform better than DAISY and SIFT for the MPI-Sintel dataset.

Full text and
other links
Department(s)University of Stuttgart, Institute of Visualisation and Interactive Systems, Visualisation and Interactive Systems
Superviser(s)Bruhn, Prof. Andrés; Maurer, Daniel
Entry dateJune 11, 2019
   Publ. Computer Science