Figure 1. An example of our new Roto++ tool working with a professional artist to increase their productivity. The artist has already specified a number of keyframes but is not satisfied with one of the intermediate frames. Under standard baselines, correcting the erroneous curve requires moving the individual control points of the spline. Using our new shape model we are able to provide an Intelligent Drag Tool that can generate likely shapes given the other keyframes. In our new interaction, the user simply selects the in correct points and drags them all towards the correct shape. Our shape model them correctly proposes the new control point locations, allowing the correction to be performed in a single operation.

Rotoscoping (cutting out different characters/objects/layers in raw video footages) is a ubiquitous task in modern post-production and represents a significant investment in person-hours. In this work, we study the particular task of professional rotoscoping for high-end, live action movies and propose a new framework that works with roto-artists to accelerate the workflow and improve their productivity.

Figure 2. The Expert Study with Top Rotoscoping Artists. In the expert study, we invite seven top artists (up to nine years experience in rotoscoping) to test our tool. Our tool provides three modes (our method and two baselines) and is able to log all the user operations by time. In this case, the artists are required to roto three shots from a commercial film then answer questionnaire. We can quantitatively evaluate the our approach.

Working with the existing keyframing paradigm, our first contribution is the development of a shape model that is updated as artists add successive keyframes. This model is used to improve the output of traditional interpolation and tracking techniques, reducing the number of keyframes that need to be specified by the artist. Our second contribution is to use the same shape model to provide a new interactive tool that allows an artist to reduce the time spent editing each keyframe. The more keyframes that are edited, the better the interactive tool becomes, accelerating the process and making the artist more efficient without compromising their control. Finally, we also provide a new, professionally rotoscoped dataset that enables truly representative, real-world evaluation of rotoscoping methods. We used this dataset to perform a number of experiments, including an expert study with professional roto-artists, to show, quantitatively, the advantages of our approach.

Update (10 Aug. 2016): Our source code has been downloaded over 1600 times since the conference. Everybody is more than welcome to contribute to Roto++ via Github or sending us emails.


  title   = {Roto++: Accelerating Professional Rotoscoping using Shape Manifolds},
  author  = {Wenbin Li and Fabio Viola and Jonathan Starck and Gabriel J. Brostow and Neill D.F. Campbell},
  journal = {ACM Transactions on Graphics (In proceeding of ACM SIGGRAPH'16)},
  volume  = {35},
  number  = {4},
  year    = {2016}


The "MODE" footage is provided courtesy of James Courtenay Media and the rotoscoping work was kindly performed for The Foundry by Pixstone Images. All data-sets were prepared by Courtney Pryce with special thanks to John Devasahayam and the PixStone Images team for turning around a vast amount of work in such a short period of time.
The study was coordinated by Juan Salazar, Dan Ring and Sara Coppola, with thanks to experts from Lipsync Post, The Moving Picture Company, and Double Negative - Kathy Toth, Andy Quinn, Benjamin Bratt, Richard Baillie, Huw Whiddon, William Dao and Shahin Toosi.
This research has received funding from the EPSRC "Centre for the Analysis of Motion, Entertainment Research and Applications" CAMERA (EP/M023281/1) and "Acquiring Complete and Editable Outdoor Models from Video and Images" OAK (EP/K023578/1, EP/K02339X/1) projects, as well as the European Commission's Seventh Framework Programme under the CR-PLAY (no 611089) and DREAMSPACE (no 610005) projects.