Roto++: Accelerating Professional Rotoscoping
using Shape Manifolds

Figure 1. An example of our new Roto++ tool working with a professional artist to increase their productivity. The artist has already specified a number of keyframes but is not satisfied with one of the intermediate frames. Under standard baselines, correcting the erroneous curve requires moving the individual control points of the spline. Using our new shape model we are able to provide an Intelligent Drag Tool that can generate likely shapes given the other keyframes. In our new interaction, the user simply selects the in correct points and drags them all towards the correct shape. Our shape model them correctly proposes the new control point locations, allowing the correction to be performed in a single operation.

Rotoscoping (cutting out different characters/objects/layers in raw video footages) is a ubiquitous task in modern post-production and represents a significant investment in person-hours. In this work, we study the particular task of professional rotoscoping for high-end, live action movies and propose a new framework that works with roto-artists to accelerate the workflow and improve their productivity.

Figure 2. The Expert Study with Top Rotoscoping Artists. In the expert study, we invite seven top artists (up to nine years experience in rotoscoping) to test our tool. Our tool provides three modes (our method and two baselines) and is able to log all the user operations by time. In this case, the artists are required to roto three shots from a commercial film then answer questionnaire. We can quantitatively evaluate the our approach.

Working with the existing keyframing paradigm, our first contribution is the development of a shape model that is updated as artists add successive keyframes. This model is used to improve the output of traditional interpolation and tracking techniques, reducing the number of keyframes that need to be specified by the artist. Our second contribution is to use the same shape model to provide a new interactive tool that allows an artist to reduce the time spent editing each keyframe. The more keyframes that are edited, the better the interactive tool becomes, accelerating the process and making the artist more efficient without compromising their control. Finally, we also provide a new, professionally rotoscoped dataset that enables truly representative, real-world evaluation of rotoscoping methods. We used this dataset to perform a number of experiments, including an expert study with professional roto-artists, to show, quantitatively, the advantages of our approach.

Update (10 Aug. 2016): Our source code has been downloaded over 1600 times since the conference. Everybody is more than welcome to contribute to Roto++ via Github or sending us emails.