UCL Logo VISUAL COMPUTING
Images from our method rendered in 1080p at 55 Hz on an Nvidia Titan X GPU. Input is an RGB-D video and 298 high-quality photos of 'Dr Johnson's house', London. With no wheelchair access to this floor, curators were keen to have their rooms digitized.

Abstract

Our aim is to give users real-time free-viewpoint rendering of real indoor scenes, captured with off-the-shelf equipment such as a high-quality color camera and a commodity depth sensor. Image-based Rendering (IBR) can provide the realistic imagery required at real-time speed. For indoor scenes however, two challenges are especially prominent. First, the reconstructed 3D geometry must be compact, but faithful enough to respect occlusion relationships when viewed up close. Second, man-made materials call for view-dependent texturing, but using too many input photographs reduces performance. We customize a typical RGB-D 3D surface reconstruction pipeline to produce a coarse global 3D surface, and local, per-view geometry for each input image. Our tiled IBR preserves quality by economizing on the expected contributions that entire groups of input pixels make to a final image. The two components are designed to work together, giving real-time performance, while hardly sacrificing quality. Testing on a variety of challenging scenes shows that our inside-out IBR scales favorably with the number of input images.


Code (coming soon) Code (soon)
Datasets (coming soon) Datasets (soon)

BibTeX

@article{InsideOut2016,
  author    = {Hedman, Peter and Ritschel, Tobias and Drettakis, George and Brostow, Gabriel},
  title     = {{Scalable Inside-Out Image-Based Rendering}},
  booktitle = {ACM Trans. Graph.},
  publisher = {ACM},
  volume    = {35},
  number    = {6},
  pages     = {231:1--231:11},
  year      = {2016}
}

Acknowledgements

Thanks to Dr Johnson's Museum, Judd Books, Waterstones on Gower Street, and Kathryn Emmett-Brostow for being so accommodating with our filming. Thanks to Rodrigo Ortiz-Cayon for computing the comparisons. The authors are grateful for the support of EU project CR-PLAY (no 611089) www.cr-play.eu, EPSRC EP/K023578/1, and project SEMAPOLIS (ANR-13-CORD-0003).