Citris Allows 3-d Interactions

UC Berkeley students may soon be able to meet professors at UC Davis in ancient Sicily for lively intellectual discussions.

Recent visual computer science advances by Ruzena Bajcsy, director of the Center for Information Technology Research in the Interest of Society (CITRIS) at UC Berkeley, may make such interactions possible.

Bajcsy's technology takes pictures of a subject in her laboratory from 48 different cameras and combines them into a 3-D image. The image can then be placed into historical Sicily, one of the three cyberspace environments created so far.

"Once you have this technology, you can ask questions about how people behave in various environments like historical Athens and Rome," Bajcsy said. "It can also help with manual physical training, any place where you need to do moving in 3-D."

Her laboratory is now recruiting scientists from multiple disciplines to ask these questions in a controlled setting. The technology is applicable to historians, social scientists, medical doctors, psychologists, educators and others.

"Bajcsy has been really visionary with all of this. We've imagined these things and she's been working with us to make them real," said David Goldberg, director of the UC system Center of Humanities, a collaborator on the project. Art historians, anthropologists and archeologists working with Goldberg have imagined a virtual museum using Bajcsy's technology where both experts and the public could virtually pick up objects and study them.

The new insights could be far-reaching. Bajcsy aims to impact common people, by studying how people behave and trust each other in cyber environments. Specifically, one could study the difference between cyberspace interaction and a face-to-face interaction, or between interactions where the whole body or just the face or hands are visualized.

This is the first time a real-time action can be perceived and manipulated in 3-D. It's one step up from 2-D television viewing, which is limited in viewpoint and ambiguous in interpreting motions. Now, a person could zoom in and out or around another person during an interaction. For example, a person could converse with someone face-to-face and jointly see from behind that he is scratching his back, due to the multiple camera viewpoints.

"TV is a 2-D image-you're stuck with whatever they give you. This is 3-D, it allows you to move through the image as if you were there," Bajcsy said.

However, the technology is still a work in progress.

"Have we solved all the problems? Far from it," Bajscy said.

Sang-Hack Jung, who completed his Ph.D. in computer science at the University of Pennsylvania where Bajcsy started the research eight years ago, said because he was wearing "bad clothes"- dark gray pants the same color as the floor of the French cathedral environment-his leg was cut off the screen.

Currently, a cable is required to transmit the large amount of information from the two sites to the digital environment. The cable is expensive and can only be used between the two sites, but Bajcsy hopes to make the technology available to many social scientists who have only meager funding.

"Digitizing (approximately) 50 cameras into the computer is not easy," said Professor Takeo Kanade, a Carnegie Mellon professor who has performed similar research. "Dealing with such large amounts of data is an enormous task-just to start the cameras you must press 50 start buttons," he said.

Funding has also been an issue, due to the holes in the images and the complicated collaborations needed between computer scientists to create the necessary computer vision, real-time graphics and computer networking. Many professors at various universities have started similar work and given up.

But Bajcsy's laboratory is forging ahead. Bajcsy left yesterday for UC Davis to begin collaborating by merging technology at both sites.

"You've always lived in a physical space," Bajcsy said. "This (technology) gives you some real options."

Tags: