UCL Logo VISUAL COMPUTING
a) Script Creation: The LookOut GUI enables a user to pre-script where the camera should point. It allows the user to define multiple alternate scripts, as seen here, and to switch between them on-the-fly. A script consists of one or more behaviors chained together. A behavior can be as simple as a pan, or as complex as positioning multiple subjects in different parts of the frame. Each camera behavior is triggered by a cue, such as the appearance of a specific actor, someone issuing a voice command, or the actor reaching a specific zone within the frame. b) Guided Setup: For field-use, LookOut resembles a very simplified dialog system, guiding the user through system checks and scene-specific initialization. The user-worn LookOut rig consists of a light backpack computer, a hand-held motorized gimbal, dual cameras (normal and wide-view), earphones, a lapel microphone, and a joystick for initial setup. When first turned on, LookOut prompts the user, through text-to-speech, to set audio levels, choose a script, and assign actor identities as needed. c-f) Interactive Filming: Four frames from a LookOut-captured video, but with false-coloring to visualize which actor(s) the scripted behaviors were attending to at the time. At the user’s instruction, LookOut frames (c) both dancers, then (d) orients the gimbal to center on the male, then the female (e), and back to the male (f). The user receives audio feedback when switching between camera behaviors. Without a field monitor, the user can watch where they’re going, while trusting our controller to handle their dynamic requests.

I want to see videos! Right here.

Abstract

The job of a camera operator is more challenging, and potentially dangerous, when filming long moving camera shots. Broadly, the operator must keep the actors in-frame while safely navigating around obstacles, and while fulfilling an artistic vision. We propose a unified hardware and software system that distributes some of the camera operator's burden, freeing them up to focus on safety and aesthetics during a take. Our real-time system provides a solo operator with end-to-end control, so they can balance on-set responsiveness to action \vs planned storyboards and framing, while looking where they're going. By default, we film without a field monitor.

Our LookOut system is built around a lightweight commodity camera gimbal mechanism, with heavy modifications to the controller, which would normally just provide active stabilization. Our control algorithm reacts to speech commands, video, and a pre-made script. Specifically, our automatic monitoring of the live video feed saves the operator from distractions. In pre-production, an artist uses our GUI to design a sequence of high-level camera "behaviors.'' Those can be specific, based on a storyboard, or looser objectives, such as "frame both actors.'' Then during filming, a machine-readable script, exported from the GUI, ties together with the sensor readings to drive the gimbal. To validate our algorithm, we compared tracking strategies, interfaces, and hardware protocols, and collected impressions from a) film-makers who used all aspects of our system, and b) film-makers who watched footage filmed using LookOut.

Main Video

Validation Video


BibTeX

@misc{sayed2020lookout,
      title={LookOut! Interactive Camera Gimbal Controller for Filming Long Takes}, 
      author={Mohamed Sayed and Robert Cinca and Enrico Costanza and Gabriel Brostow},
      year={2020},
      eprint={2012.02087},
      archivePrefix={arXiv},
      primaryClass={cs.GR}
      }