15-05-2012, 11:36 AM
Technologies for Virtual Reality/Tele-Immersion Applications:
Issues of Research in Image Display and Global Networking
[attachment=22064]
Abstract
The Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago (UIC) has developed an
aggressive program over the past decade to partner with scores of computational scientists and engineers all over the
world. The focus of this effort has been to create visualization and virtual reality (VR) devices and applications for
collaborative exploration of scientific and engineering data. Since 1995, our research and development activities
have incorporated emerging high bandwidth networks like the vBNS and its international connection point STAR
TAP, in an effort now called tele-immersion.
As a result of eight years’ experience building first and second-generation projection-based VR devices to support
these applications, we wish to describe needed research in third-generation VR devices aimed at desktop/officesized
displays. Since no current projection technology is yet configurable with ideal resolution and size, we must first
describe the variety of emerging display devices, such as large color plasma displays, LCD projectors, LED panels,
Digital Light Valves (DLVs), Grating Light Valves (GLVs), and Digital Micro Mirror Displays (DMDs ).
Challenges of Tele-Immersion
Tele-immersion has emerged as a high-end driver for the Quality of Service (QoS), bandwidth, and reservation
efforts envisioned by the NGI and Internet2 leadership. From a networking perspective, tele-immersion is a very
challenging technology for several reasons:
• The networks must be in place and tuned to support high-bandwidth applications
• Low latency, needed for 2-way collaboration, is hard to specify and guarantee given current middleware
• The speed of light in fiber itself is a limiting factor over transcontinental and transoceanic distances
• Multicast, unicast, reliable and unreliable data transmissions (called “flows”) need to be provided for and
managed by the networks and the operating systems of supercomputer-class workstations
• Real-time considerations for video and audio reconstruction (“streaming”) are critical to achieving the feel
of telepresence, whether synchronous or recorded and played back
• The computers, too, are bandwidth limited with regard to handling very large data for collaboration
• Simulation and data mining are open-ended in computational and bandwidth needs—there will never be
quite enough computing and bits/second to fully analyze, and simulate reality for scientific purposes
Tele-Immersion Flow Types
Progress in all these areas, however, is expected; tele-immersion serves as an integrating technology as pieces of the
solution are contributed by the community and computer/networking industry. The following table, developed in
discussions with Rick Stevens, director of the Math and Computer Science Division at Argonne National Lab, gives
EC/NSF Workshop on Research Frontiers in Virtual Environments and Human-Centered Computing 5
our current best estimations and opinions of the attributes of the nine types flows simultaneously needed for an nway
compute and data-intensive audio, video, and haptic (touch) tele-immersive session. [35] The research agenda
for the coming years very much involves validating this table and creating software intelligence to compensate for
the otherwise unachievable.
Lag vs. Network Latency
In tele-immersion systems, an important distinction must be made between the notions of network performance and
user perceived lag in the virtual environment. “Lag” is the term used to describe the perceived sum of all the sources
of latency in a system.
Typically, it is thought of as the delay between action in the real world (e.g., as captured by tracking or haptics) and
the perceived response of the system to that action. Lag is the critical issue for usability; reducing lag is a major
technical challenge. Communications latency is only one component of tele-immersion lag. Effective solutions to
reducing lag must attack the component sources of latency at all levels of the system. VR system lag is the result of
delays in rendering, display, tracking, simulation, communications, and synchronization. There are multiple sources
of latency in the communications system alone:
Issues of Research in Image Display and Global Networking
[attachment=22064]
Abstract
The Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago (UIC) has developed an
aggressive program over the past decade to partner with scores of computational scientists and engineers all over the
world. The focus of this effort has been to create visualization and virtual reality (VR) devices and applications for
collaborative exploration of scientific and engineering data. Since 1995, our research and development activities
have incorporated emerging high bandwidth networks like the vBNS and its international connection point STAR
TAP, in an effort now called tele-immersion.
As a result of eight years’ experience building first and second-generation projection-based VR devices to support
these applications, we wish to describe needed research in third-generation VR devices aimed at desktop/officesized
displays. Since no current projection technology is yet configurable with ideal resolution and size, we must first
describe the variety of emerging display devices, such as large color plasma displays, LCD projectors, LED panels,
Digital Light Valves (DLVs), Grating Light Valves (GLVs), and Digital Micro Mirror Displays (DMDs ).
Challenges of Tele-Immersion
Tele-immersion has emerged as a high-end driver for the Quality of Service (QoS), bandwidth, and reservation
efforts envisioned by the NGI and Internet2 leadership. From a networking perspective, tele-immersion is a very
challenging technology for several reasons:
• The networks must be in place and tuned to support high-bandwidth applications
• Low latency, needed for 2-way collaboration, is hard to specify and guarantee given current middleware
• The speed of light in fiber itself is a limiting factor over transcontinental and transoceanic distances
• Multicast, unicast, reliable and unreliable data transmissions (called “flows”) need to be provided for and
managed by the networks and the operating systems of supercomputer-class workstations
• Real-time considerations for video and audio reconstruction (“streaming”) are critical to achieving the feel
of telepresence, whether synchronous or recorded and played back
• The computers, too, are bandwidth limited with regard to handling very large data for collaboration
• Simulation and data mining are open-ended in computational and bandwidth needs—there will never be
quite enough computing and bits/second to fully analyze, and simulate reality for scientific purposes
Tele-Immersion Flow Types
Progress in all these areas, however, is expected; tele-immersion serves as an integrating technology as pieces of the
solution are contributed by the community and computer/networking industry. The following table, developed in
discussions with Rick Stevens, director of the Math and Computer Science Division at Argonne National Lab, gives
EC/NSF Workshop on Research Frontiers in Virtual Environments and Human-Centered Computing 5
our current best estimations and opinions of the attributes of the nine types flows simultaneously needed for an nway
compute and data-intensive audio, video, and haptic (touch) tele-immersive session. [35] The research agenda
for the coming years very much involves validating this table and creating software intelligence to compensate for
the otherwise unachievable.
Lag vs. Network Latency
In tele-immersion systems, an important distinction must be made between the notions of network performance and
user perceived lag in the virtual environment. “Lag” is the term used to describe the perceived sum of all the sources
of latency in a system.
Typically, it is thought of as the delay between action in the real world (e.g., as captured by tracking or haptics) and
the perceived response of the system to that action. Lag is the critical issue for usability; reducing lag is a major
technical challenge. Communications latency is only one component of tele-immersion lag. Effective solutions to
reducing lag must attack the component sources of latency at all levels of the system. VR system lag is the result of
delays in rendering, display, tracking, simulation, communications, and synchronization. There are multiple sources
of latency in the communications system alone: