Video normals from colored lights

Gabe Brostow, Carlos Hernández, George Vogiatzis, Björn Stenger, Roberto Cipolla

Research output: Contribution to journalArticle

Abstract

We present an algorithm and the associated single-view capture methodology to acquire the detailed 3D shape, bends, and wrinkles of deforming surfaces. Moving 3D data has been difficult to obtain by methods that rely on known surface features, structured light, or silhouettes. Multispectral photometric stereo is an attractive alternative because it can recover a dense normal field from an untextured surface. We show how to capture such data, which in turn allows us to demonstrate the strengths and limitations of our simple frame-to-frame registration over time. Experiments were performed on monocular video sequences of untextured cloth and faces with and without white makeup. Subjects were filmed under spatially separated red, green, and blue lights. Our first finding is that the color photometric stereo setup is able to produce smoothly varying per-frame reconstructions with high detail. Second, when these 3D reconstructions are augmented with 2D tracking results, one can register both the surfaces and relax the homogenous-color restriction of the single-hue subject. Quantitative and qualitative experiments explore both the practicality and limitations of this simple multispectral capture system.
Original languageEnglish
Pages (from-to)2104-2114
Number of pages11
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume33
Issue number10
DOIs
Publication statusPublished - Oct 2011

Fingerprint

Photometric Stereo
Color
Blue Light
Structured Light
Silhouette
3D shape
3D Reconstruction
Registration
Experiment
Data acquisition
Experiments
Restriction
Methodology
Alternatives
Demonstrate

Bibliographical note

© 2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Keywords

  • photometric stereo
  • multispectral
  • single view
  • video normals

Cite this

Brostow, Gabe ; Hernández, Carlos ; Vogiatzis, George ; Stenger, Björn ; Cipolla, Roberto. / Video normals from colored lights. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. 2011 ; Vol. 33, No. 10. pp. 2104-2114 .
@article{3977e683d54c4123b5d8fe04adf4e3c1,
title = "Video normals from colored lights",
abstract = "We present an algorithm and the associated single-view capture methodology to acquire the detailed 3D shape, bends, and wrinkles of deforming surfaces. Moving 3D data has been difficult to obtain by methods that rely on known surface features, structured light, or silhouettes. Multispectral photometric stereo is an attractive alternative because it can recover a dense normal field from an untextured surface. We show how to capture such data, which in turn allows us to demonstrate the strengths and limitations of our simple frame-to-frame registration over time. Experiments were performed on monocular video sequences of untextured cloth and faces with and without white makeup. Subjects were filmed under spatially separated red, green, and blue lights. Our first finding is that the color photometric stereo setup is able to produce smoothly varying per-frame reconstructions with high detail. Second, when these 3D reconstructions are augmented with 2D tracking results, one can register both the surfaces and relax the homogenous-color restriction of the single-hue subject. Quantitative and qualitative experiments explore both the practicality and limitations of this simple multispectral capture system.",
keywords = "photometric stereo, multispectral, single view, video normals",
author = "Gabe Brostow and Carlos Hern{\'a}ndez and George Vogiatzis and Bj{\"o}rn Stenger and Roberto Cipolla",
note = "{\circledC} 2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.",
year = "2011",
month = "10",
doi = "10.1109/TPAMI.2011.37",
language = "English",
volume = "33",
pages = "2104--2114",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
issn = "0162-8828",
publisher = "IEEE",
number = "10",

}

Video normals from colored lights. / Brostow, Gabe; Hernández, Carlos; Vogiatzis, George; Stenger, Björn; Cipolla, Roberto.

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 10, 10.2011, p. 2104-2114 .

Research output: Contribution to journalArticle

TY - JOUR

T1 - Video normals from colored lights

AU - Brostow, Gabe

AU - Hernández, Carlos

AU - Vogiatzis, George

AU - Stenger, Björn

AU - Cipolla, Roberto

N1 - © 2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

PY - 2011/10

Y1 - 2011/10

N2 - We present an algorithm and the associated single-view capture methodology to acquire the detailed 3D shape, bends, and wrinkles of deforming surfaces. Moving 3D data has been difficult to obtain by methods that rely on known surface features, structured light, or silhouettes. Multispectral photometric stereo is an attractive alternative because it can recover a dense normal field from an untextured surface. We show how to capture such data, which in turn allows us to demonstrate the strengths and limitations of our simple frame-to-frame registration over time. Experiments were performed on monocular video sequences of untextured cloth and faces with and without white makeup. Subjects were filmed under spatially separated red, green, and blue lights. Our first finding is that the color photometric stereo setup is able to produce smoothly varying per-frame reconstructions with high detail. Second, when these 3D reconstructions are augmented with 2D tracking results, one can register both the surfaces and relax the homogenous-color restriction of the single-hue subject. Quantitative and qualitative experiments explore both the practicality and limitations of this simple multispectral capture system.

AB - We present an algorithm and the associated single-view capture methodology to acquire the detailed 3D shape, bends, and wrinkles of deforming surfaces. Moving 3D data has been difficult to obtain by methods that rely on known surface features, structured light, or silhouettes. Multispectral photometric stereo is an attractive alternative because it can recover a dense normal field from an untextured surface. We show how to capture such data, which in turn allows us to demonstrate the strengths and limitations of our simple frame-to-frame registration over time. Experiments were performed on monocular video sequences of untextured cloth and faces with and without white makeup. Subjects were filmed under spatially separated red, green, and blue lights. Our first finding is that the color photometric stereo setup is able to produce smoothly varying per-frame reconstructions with high detail. Second, when these 3D reconstructions are augmented with 2D tracking results, one can register both the surfaces and relax the homogenous-color restriction of the single-hue subject. Quantitative and qualitative experiments explore both the practicality and limitations of this simple multispectral capture system.

KW - photometric stereo

KW - multispectral

KW - single view

KW - video normals

UR - http://www.scopus.com/inward/record.url?scp=80051997968&partnerID=8YFLogxK

UR - http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=5719620

U2 - 10.1109/TPAMI.2011.37

DO - 10.1109/TPAMI.2011.37

M3 - Article

VL - 33

SP - 2104

EP - 2114

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

IS - 10

ER -