Human-Computer Interaction


We’re interested in the many ways computer interfaces can span the digital and tangible worlds, giving rise to qualitatively new experiences. Our agenda takes advantage of technologies that are relatively new in the commercial world, and whose interactions have not yet been fully explored. Our researchers invent new technologies for sensing touch and pose, as well as creating new sensory experiences such as haptic illusions.


(in alphabetical order)

AIREAL: Interactive Tactile Experiences in Free Air

AIREAL is a low cost, scalable haptic technology that delivers rich tactile sensations in mid air. AIREAL enables users to feel virtual objects without requiring the user to wear or touch a physical device. To deliver these tactile sensations, AIREAL uses a vortex, a ring of air that can impart a significant force the user can feel at large distances. The AIREAL technology is almost entirely 3D printed including an enclosure, flexible nozzle and a pan and tilt gimbal structure that allows a vortex to be precisely delivered to any location in 3D space. We envision using AIREAL in numerous applications including gaming and storytelling, location based entertainment and simulation applications.

Botanicus Interacticus: Interactive Plant Technology

Botanicus Interacticus is a technology for designing highly expressive interactive plants, both living and artificial. Driven by the rapid fusion of our computing and living spaces, Botanicus Interacticus takes interaction from computing devices and places it anywhere where living plants are present. The interaction is rich and extensive, it goes beyond simple touch detection and allows complex gestural interaction, such as sliding fingers on the stem of the plant, detecting touch and grasp location, tracking proximity between the user hand and a plant, and estimating the amount of touch contact.

Capacitive Fingerprinting

We propose a novel human identification and differentiation techniques based on Touché sensing technology. Touché measures the impedance of a user to the environment by sweeping across a range of AC frequencies. Different people have different bone densities and muscle mass, wear different clothes and footwear, and so on. This, in turn, yields different impedance profiles, which allows attribute touch events and multitouch gestures to a particular user and modify interaction and system response accordingly. Note that this technology does not require any instrumentation of the user or the environment. We refer to this technology as Capacitive Fingerprinting.

Interactive Mobile Projectors

Mobile projectors are small projectors that can be embedded in handheld computing devices such as smartphones and tablets. Market research predicts that as many as 39 million devices with embedded projectors will be on the market by 2014. Developing new interfaces for mobile projectors opens up a range of possibilities for interactive experiences in both work and play. (Projects included: MotionBeam; SideBySide)

Ishin-Den-Shin: Transmitting Sound Through Touch

Ishin-Den-Shin explores the use of the human body as sound transmission medium. The technology turns an audio message into an inaudible signal that is relayed by the human body. When the communicator’s finger slightly rubs an object, this physical interaction creates an ad-hoc speaker that makes it possible to hear the recorded sounds. A special case of Ishin-Den-Shin interactioh happens when the communicator touches another person’s ear. In this case, a modulated electrostatic field creates a very small vibration of the ear lobe; the finger and the other person’s ear, together, form a speaker which makes the signal audible only for the person touched.

Paper Generators: Harvesting Energy from Touching, Rubbing and Sliding

We present a new energy harvesting technology that generates electrical energy from a user’s interactions with paper-like materials. The energy harvesters are flexible, light, and inexpensive, and they utilize a user’s gestures such as tapping, touching, rubbing and sliding to generate energy. The harvested energy is then used to actuate LEDs, e-paper displays and other devices to create interactive applications for books and other printed media.


PAPILLON is a technology for 3D printing highly expressive animated eyes for interactive characters, robots and toys. Expressive eyes are essential in any form of face-to-face communication and designing them has been a critical challenge in robotics, as well as in interactive character and toy development. Crucially, the traditional animatroncis approach is not applicable to fictional characters from animated movies, comics and cartoons whose eye expressions are non-realistic, highly exaggerated and can take any size and shape to communicate a character’s emotions and intentions, e.g. “dollar signs” for greed or a “heart” for romance. [Projects include: Expressive Eyes for Interactive Characters (ACM SIGGRAPH 2013) -and- Designing Curved Display Surfaces with Printed Optics (ACM UIST 2013)]

Printed Optics

Printed Optics is a new approach to creating custom optical elements for interactive devices using 3D printing. Printed Optics enable sensing, display, and illumination elements to be directly embedded in the body of an interactive device. Using these elements, unique display surfaces, novel illumination techniques, custom optical sensors, and robust embedded components can be digitally fabricated for rapid, high fidelity, customized interactive devices.

Revel: Programming the Sense of Touch

REVEL is a new wearable tactile technology that modifies the user’s tactile perception of the physical world. Current tactile technologies enhance objects and devices with various actuators to create rich tactile sensations, limiting the experience to the interaction with instrumented devices. In contrast, REVEL can add artificial tactile sensations to almost any surface or object, with very little if any instrumentation of the environment. As a result, REVEL can provide dynamic tactile sensations on touch screens as well as everyday objects and surfaces in the environment, such as furniture, walls, wooden and plastic objects, and even human skin.

Surround Haptics

Surround Haptics is a new tactile technology that uses a low- resolution grid of inexpensive vibrating actuators to generate high- resolution, continuous, moving tactile strokes on human skin. It is based on a carefully designed and thoroughly evaluated algorithm that uses tactile illusions to create and move virtual actuators anywhere on a grid. The technology has implications in the video games, movies, rides, toys, consumer products, medical devices, assistive devices, sporting equipments and more. (Projects include: Tactile Brush; Immersive Tactile Experiences)

Tactile Rendering of 3D Features on Touch Surfaces

In this project, we develop and apply a tactile rendering algorithm to simulate rich 3D geometric features (such as bumps, ridges, edges, protrusions, texture etc.) on touch screen surfaces. The underlying hypothesis is that when a finger slides on an object then minute surface variations are sensed by friction-sensitive mechanoreceptors in the skin. Thus, modulating the friction forces between the fingertip and the touch surface would create illusion of surface variations. We propose that the perception of a 3D “bump” is created when local gradients of the virtual bump are mapped to lateral friction forces.

Tesla Touch

We present a new technology for enhancing touch interfaces with tactile feedback. The proposed technology is based on the electrovibration principle, does not use any moving parts and provides a wide range of tactile feedback sensations to fingers moving across a touch surface. When combined with an interactive display and touch input, it enables the design of a wide variety of interfaces that allow the user to feel virtual elements through touch.

A Tongue Input Device for Creating Conversations

We present a new tongue input device, the tongue joystick, for use by an actor inside an articulated-head character costume. Using our device, the actor can maneuver through a dialogue tree, selecting clips of prerecorded audio to hold a conversation in the voice of the character.


Touché is a new sensing technology that proposes a Swept Frequency Capacitive Sensing technique that can not only detect a touch event, but simultaneously recognize complex configurations of the human fingers, hand and body during touch interaction. This allows to significantly enhances touch interfaces in a broad range of applications, from conventional touchscreens to designing novel interaction scenarios for unique contexts and materials, such as human body and liquids.