Computer Vision

Guest interaction at theme parks, motion capture for studios, and sports visualization are just a few of the direct applications for our computer-vision research. We also perform research in which computer vision intersects with human-computer interaction, video processing, display technology, and optics: it plays a role in our work on input devices, content-aware video processing, projector-camera systems, and computational cinematography.

Projects

Activity Map

Activity Map is an applied project on tracking people and characterizing the appearance of individuals. Applications of the technology include both logistical support to ensure that environments with people are functioning correctly, and entertainment such as interactivity.

Current work is on tracking people in indoor environments using a multi-camera system, using a probabilistic framework that incorporates human detectors. Appearance profiles are captured for individuals to enable reacquisition after occlusion, and reacquisition across different sensor deployments. We are augmenting the vision system with Kinect depth sensors, a low-cost technology with potential to radically impact indoor tracking. This work on people tracking complements other work in the computer vision group on mobile robot localization, with a longer-term vision of providing safety monitoring when people and mobile robots are sharing the same space

High-Quality Single-Shot Capture of Facial Geometry

This project develops a passive stereo system for capturing the 3D geometry of a face in a single-shot with standard light sources. The system is low-cost and easy to deploy. Results are sub-millimeter accurate and commensurate with those from state-of-the-art systems based on active lighting (e.g., laser scanners), and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography. [More]

Multi-Sensor Fusion Cam*

Traditional cinematographic cameras consist of a single camera, while two-camera cinematographic rigs have also become common with the recent wave of 3D cinema. Taking camera design a further step, this project proposes a system in which a central cinematographic camera is augmented with a clip-on frame of satellite sensors. The satellite devices include compact cameras, a depth sensor, and a thermal camera. The result is a FusionCam that supports more powerful post-production analysis than is possible with a single camera or a two-camera rig, and is able to synthetically generate stereoscopic 3D imagery with specified stereo parameters.

The core research challenge is to produce better depth maps by integrating the high-resolution image from the central cinematographic camera with the information from the satellite sensor modalities. Performing fusion of these different modalities raises questions regarding how the strengths of the modalities can be best exploited, and how the weaknesses of each can best be compensated for. Current work is on analysis of the data at a single time instance, and new work will extend this to temporal analysis of video. [More...]

*This project is categorized under Video Processing and Computer Vision.

Point-less Calibration

Point-based targets, such as checkerboards, are often not practical for outdoor camera calibration, as cameras are usually at significant heights requiring extremely large calibration patterns on the ground. Fortunately, it is possible to make use of existing non-point landmarks in the scene by formulating camera calibration in terms of image alignment. In this paper, we simultaneously estimate the camera intrinsic, extrinsic and lens distortion parameters directly by aligning to a planar schematic of the scene. [More...]

Procams Toolbox

The setup and control of projectors in a projector installation is increasingly automatic. Some examples of automated processing are the keystone-correction found in most consumer projectors, multi-projector systems that combine several overlapping projections to create a super-image, and projection onto 3D objects such as buildings with automatic compensation for object shape.

Procams Toolbox is an applied project on a software toolbox that can be used to create a range of applications for different projector deployments with varying requirements. The core of the toolbox is a set of standard software components such as projector calibration, camera calibration, screen calibration, and similar. These components are modular and are plugged together in a universal application framework. The toolbox is being grown into a resource that is useful for any project installation, with research anticipated in projection onto 3D objects and dynamic scenes.