16-05-2012, 12:02 PM
Technologies for Virtual Reality/Tele-Immersion Applications: Issues of Research in Image Display and Global Networking
Technologies for Virtual RealityTele-Immersion Applications.pdf (Size: 977.13 KB / Downloads: 83)
1. Abstract
The Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago (UIC) has developed an
aggressive program over the past decade to partner with scores of computational scientists and engineers all over the
world. The focus of this effort has been to create visualization and virtual reality (VR) devices and applications for
collaborative exploration of scientific and engineering data. Since 1995, our research and development activities
have incorporated emerging high bandwidth networks like the vBNS and its international connection point STAR
TAP, in an effort now called tele-immersion.
As a result of eight years’ experience building first and second-generation projection-based VR devices to support
these applications, we wish to describe needed research in third-generation VR devices aimed at desktop/officesized
displays. Since no current projection technology is yet configurable with ideal resolution and size, we must first
describe the variety of emerging display devices, such as large color plasma displays, LCD projectors, LED panels,
Digital Light Valves (DLVs), Grating Light Valves (GLVs), and Digital Micro Mirror Displays (DMDs ).
EC/NSF Workshop on Research Frontiers in Virtual Environments and Human-Centered Computing, June 1-4, 1999, Chateau de Bonas, France2
In 1991, we conceived and over several years developed the CAVE virtual reality theater, a room-sized, highresolution,
projection-based system that enables users to experience excellent immersion in full 3D imagery. We then
developed the ImmersaDesk, a smaller, software-compatible, drafting-table-format version of the CAVE that has
been deployed to dozens of locations, nationally and internationally, at government institutions, national laboratories,
universities, and companies.
The hardware now needs to be made smaller, higher resolution and more adaptable to the human and his/her
workspace. Middleware that manages connections, bandwidth and latency needs to be integrated with the computer
systems driving these hardware devices. Software that increases the quality of human-computer interaction through
human output recognition must be brought from specialized lab experiments to routine use, and provided as part of
the tele-immersive collaborative experience. This paper discusses many of the issues at the heart of this research.
2. Issues
2.A. Background: Projection-Based VR Technologies
The CAVE™ is a multi-person, room-sized,
high-resolution, 3D video and audio
environment. Graphics are projected in stereo
onto three walls and the floor, and viewed with
stereo glasses. As a viewer wearing a location
sensor moves within its display boundaries, the
correct perspective and stereo projections of
the environment are constantly updated, so the
image moves with and surrounds the viewer to
achieve immersion.
The ImmersaDesk™ is a drafting-table
format version of the CAVE. When folded
up, it fits through a standard institutional
door, and deploys into a 6’ x 8’ footprint. It
requires a single graphics engine of the SGI
Onyx or Octane class, one projector, and no
architectural modifications to the working
space. The ImmersaDesk is softwarecompatible
with the CAVE library.
The Infinity Wall is derivative of the Power-
Wall, a research effort of Paul Woodward at
the University of Minnesota. The PowerWall
achieves very high display resolution through
parallelism, building up a single image from an
array of display panels projected from the rear
onto a single screen. High-speed playback of
previously rendered images is possible by
attaching extremely fast disk subsystems,
accessed in parallel, to an Onyx. The Infinity
Wall is a simpler PowerWall that has tracking
and stereo; it is CAVE library compatible.
Computational Science and Engineering Research Partners. Since 1986, EVL has partnered with the National Center
for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign and the Mathematics
and Computer Science Division at Argonne National Laboratory in ongoing efforts to develop national
collaborations at professional conferences—notably ACM SIGGRAPH and ACM/IEEE Supercomputing. These
events emphasize high performance computing and communications, VR, and scientific visualization. The overall
purpose is to encourage the development of teams, tools, hardware, system software, and human interface models on
an accelerated schedule to enable national-scale, multi-site collaborations applied to National Challenge and Grand
Challenge problems. As a result of I-WAY, an experimental high-performance network linking dozens of the USA’s
fastest computers and advanced visualization environments at Supercomputing 95, many successful CAVE
collaborations resulted. [4, 5, 16] Dozens of these scientists continue to work with EVL on various joint research
projects, either informally, through grants, or through affiliation in the NSF Partnerships for Advanced
Computational Infrastructure (PACI) program. [http://alliance.ncsa.uiuc.edu] A number have CAVEs, ImmersaDesks
and similar devices. The CAVE Research Network Users’ Society (CAVERNUS) has been in operation for several
years, and welcomes members interested in projection-based VR.[www.ncsa.uiuc.edu/VR/cavernus]
International STAR TAP Partners. STAR TAP is a persistent infrastructure to facilitate the long-term interconnection
and interoperability of advanced international networking in support of applications, performance measuring, and
technology evaluations. [www.startap.net] STAR TAP, in Chicago, is the Next Generation Internet Exchange
(NGIX) point for Next Generation Internet (NGI) [www.ngi.gov] and Internet2 [www.internet2.edu] networks.
Several institutions in foreign countries own CAVEs and ImmersaDesks or similar devices and have either obtained
EC/NSF Workshop on Research Frontiers in Virtual Environments and Human-Centered Computing 3
STAR TAP connectivity or are applying for it in order to test broadband VR collaborations. Countries currently
connected to STAR TAP include Australia, Canada, Japan, Korea, Taiwan, Singapore, Russia, Norway, Sweden,
Denmark, Iceland, Finland, The Netherlands, France, and Israel. CERN is also connected.
Tele-immersion. The term tele-immersion was first used in October 1996 as the title of a workshop organized by EVL
and sponsored by Advanced Network & Services, Inc. to bring together researchers in distributed computing,
collaboration, VR, and networking. At this workshop, we paid specific attention to the future needs of applications in
the sciences, engineering, and education. We define tele-immersion as the union of networked VR and video in the
context of significant computing and data mining. EVL’s Web site [www.evl.uic.edu] has an extensive teleimmersion
bibliography and papers. Tele-immersion has since entered the NGIand Internet2 vocabulary. In the
applications section of the Computing Research Association’s “Research Challenges for the NGI,” tele-immersion
was one of five key technologies identified as necessary for the future use of the NGI [18]:
Tele-immersion. Tele-immersion will enable users in different locations to collaborate in a shared, virtual,
or simulated environment as if they are in the same room. It is the ultimate synthesis of networking and
media technologies to enhance collaborative environments. Tele-Immersive applications must combine
audio, video, virtual worlds, simulations, and many other complex technologies. They will require huge
bandwidth, very fast responses, and guarantees of delivery.
We have connected CAVEs and ImmersaDesks over networks, from ATM-based 622 Mb and 155 Mb networks to
ISDN. We have implemented video and audio over the networks to enable users to conduct remote teleconferencing
and distributed virtual prototyping. At Supercomputing ‘97, we held a 17-way ImmersaDesk/CAVE tele-immersion
experiment with 8 ImmersaDesks on the conference exhibit floor and another 9 devices connected from as far away
as Amsterdam and Tokyo. [7] At Supercomputing’98, 10 countries participated in the iGrid booth showing many
instances of international tele-immersion. [23, 24] [www.startapigrid]
CAVERN is our acronym for the CAVE Research Network. CAVERN is comprised of dozens of network-connected
CAVEs, ImmersaDesks, and other VR devices, like Head-Mounted Displays, Responsive Workbenches, and
BOOMs. CAVERN is managed by the CAVE libraries and CAVERNsoft, a distributed shared memory software
package optimized for networked collaboration. [12, 13, 14]
2.B. Issues in Tele-Immersion Development
The ideal tele-immersion system is not hard to imagine. Combine the best computer graphics, audio, computer
simulation, and imaging. Connect with networking as good as direct memory access. Provide software and
hardware to track gaze, gesture, facial expression, and body position. Offer it as a built-in feature on all personal
computers and video games. Obviously, we are far from achieving ubiquitous tele-immersion.
Consider human voice and audio in general. There is a worldwide network optimized for speech (the telephone
system) that supports 2-way and multi-way interactions. Computers and other equipment one can purchase in
shopping malls can record, edit, playback, and duplicate (even net broadcast) audio to perfection. Real-time speech
synthesis is close at hand with gigaflop desktop machines. Similarly, for standard video, recording, editing, playback,
global teleconferencing, and broadcast, mature and optimized systems exist, at much higher cost.
No such consumer/corporate demand exists yet for tele-immersion; however, the near-term ubiquity of 3D graphics
engines, expected implosion of telecommunications costs, and emergence of new display technologies are reasons
for timely experimental development of integrated systems. We hope to inspire private sector investment by
describing prototypes of fully integrated tele-immersion hardware and software. Many of the barriers are marketbased,
but several are true technical research issues. Below, we identify and propose to address a set of these
research issues.
The tele-immersion system of 2009 would ideally:
• Support one or more flat panels/projectors with ultra-high color resolution (say 5000x5000)
• Be stereo capable without special glasses
• Have several built-in micro-cameras and microphones
• Have tether-less, low-latency, high-accuracy tracking
EC/NSF Workshop on Research Frontiers in Virtual Environments and Human-Centered Computing 4
• Network to teraflop computing via multi-gigabit optical switches with low latency
• Have exquisite directional sound capability
• Be available in a range of compatible hardware and software configurations
• Have gaze-directed or gesture-directed variable resolution and quality of rendering
• Incorporate AI-based predictive models to compensate for latency and anticipate user transitions
• Use a range of sophisticated haptic devices to couple to human movement and touch
• Accommodate disabled and fatigued users in the spirit of the Every Citizen Interface to the NII [2]
What we have as parts to integrate into 1999 systems are:
• Heavy, moderately expensive 3-tube projectors as the only straightforward stereo-capable projection
devices
• Large projection distances needed for rear projection
• Medium resolution (1280x1024 pixel) displays
• Moderately awkward stereo glasses with limited view angle
• Stereo graphics hardware that integrates poorly with non-stereo camera input
• Imprecise electromagnetic tethered tracking with significant latency
• “Best effort” networking with random latency
• Expensive multi-processor workstations and rendering engines ($25,000-$200,000/screen)
• Primitive software models of user interactions within VR and tele-immersive systems
• Very primitive hardware devices for haptic interaction
In addition to the obvious dependency on improvements in display devices, computing hardware, and network
integration, the tele-immersion system of 2009 will need to rely on emerging results from the computer science
research community, including specifically:
• Data intensive computing and data mining
• Image-based modeling
• Digital audio/video transmission
• Recording/playback of sessions
• Every citizen interfaces
• Gesture, speech, and gaze interaction