Introduction

News and event

System Architecture

Video Demos

Research Publications

Contact

Home

Body Brush is a real-time body-driven immersive environment, which is achieved with the development of a low-cost computer-vision-based motion analysis system using frontal infrared illumination, and an innovative graphic rendering software that maps the body motion gesture-path-energy to the audio-visual attributes. It captures 3D human motion and transforms the motion data into a rich variety of 3D visual forms visualized through a stereo projector, as well as 3D sound and music with a surround sound system.

Figure 1: The conceptual design of Body-Brush

Brief Introduction:

This work is inspired by the "action painting" of Jackson Pollock. Imagine when one enters a room, a room that is filled with 3-dimensional arrays of colour-path-brushstroke representations, his moving body as a whole, which is treated as a dynamic body-brush, will automatically pick up colours and leave colour marks in the 3-D canvas space in real-time. This is achieved with the development of an innovative computer vision based motion analysis system. It involves the use of frontal infrared illumination and the economy of two cameras, together with a locally developed advanced motion analysis software to extract the 3-dimensional gesture of the human body. The virtual environment it produced is inexpensive, portable and above all, totally immersive (i.e., you do not have to wear anything to be recognized by the sensor), when compared with many extremely expensive, user-unfriendly commercial systems.

Figure 2 : Jackson Pollock's "Action Painting"

The "Body Brush" has also transformed the ways of rendering, perceiving, and preserving painting. The output from the machine is a spreading of abstract, watercolour like, calligraphic brushstrokes in the 3-D canvas space. But this is only the default rendering in the programming design. In fact, the body-brush data can be processed to construct multiple brush works and composite architectural spaces. Moreover, the interface allows audience to jump into the 3-D painting and navigate from literally infinite angles and perspective ratios. Meanwhile, the interactive process of the user “painter” in the 3-D canvas, as well as the viewing patterns of the audience can be stored in the data bank. The project can thus provide valuable raw 3-D data for the study of aesthetics, human expressive and cognitive patterns, and perceptual computing.

This research is a collaborative work by the artist Young Hay and the Department of the Computer Science (CS), City University of Hong Kong, under the direction of Professor Horace Ip. The project is itself an interesting interface that integrates ideas from art and science. This work demonstrates the way science and art can work together to achieve a new vision.