Using multiple hypotheses to improve depth-maps for multi-view stereo

Neill D.F. Campbell, George Vogiatzis, Carlos Hernández, Roberto Cipolla

Research output: Chapter in Book/Report/Conference proceedingConference publication

Abstract

We propose an algorithm to improve the quality of depth-maps used for Multi-View Stereo (MVS). Many existing MVS techniques make use of a two stage approach which estimates depth-maps from neighbouring images and then merges them to extract a final surface. Often the depth-maps used for the merging stage will contain outliers due to errors in the matching process. Traditional systems exploit redundancy in the image sequence (the surface is seen in many views), in order to make the final surface estimate robust to these outliers. In the case of sparse data sets there is often insufficient redundancy and thus performance degrades as the number of images decreases. In order to improve performance in these circumstances it is necessary to remove the outliers from the depth-maps. We identify the two main sources of outliers in a top performing algorithm: (1) spurious matches due to repeated texture and (2) matching failure due to occlusion, distortion and lack of texture. We propose two contributions to tackle these failure modes. Firstly, we store multiple depth hypotheses and use a spatial consistency constraint to extract the true depth. Secondly, we allow the algorithm to return an unknown state when the a true depth estimate cannot be found. By combining these in a discrete label MRF optimisation we are able to obtain high accuracy depth-maps with low numbers of outliers. We evaluate our algorithm in a multi-view stereo framework and find it to confer state-of-the-art performance with the leading techniques, in particular on the standard evaluation sparse data sets.

Original languageEnglish
Title of host publicationComputer Vision - ECCV 2008 - 10th European Conference on Computer Vision, Proceedings
PublisherSpringer
Pages766-779
Number of pages14
EditionPART 1
ISBN (Print)3540886818, 9783540886815
DOIs
Publication statusPublished - 2008
Event10th European Conference on Computer Vision, ECCV 2008 - Marseille, France
Duration: 12 Oct 200818 Oct 2008

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume5302 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference10th European Conference on Computer Vision, ECCV 2008
CountryFrance
CityMarseille
Period12/10/0818/10/08

Fingerprint Dive into the research topics of 'Using multiple hypotheses to improve depth-maps for multi-view stereo'. Together they form a unique fingerprint.

Cite this