17-03-2014, 10:11 PM
ABSTRACT
Mid-air direct-touch interaction in stereoscopic display environments
poses challenges to the design of 3D user interfaces. Not
only is passive haptic feedback usually absent when selecting a virtual
object displayed with positive or negative parallax relative to
a display surface, but such setups also suffer from inherent visual
conflicts, such as vergence/accommodation mismatches and double
vision. In particular, if the user tries to select a virtual object with
a finger or input device, either the virtual object or the user’s finger
will appear blurred, resulting in an ambiguity for selections that
may significantly impact the user’s performance.
In this paper we evaluate the effect of visual conflicts for mid-air
3D selection performance within arm’s reach on a stereoscopic table
with a Fitts’ Law experiment. We compare three different techniques
with different levels of visual conflicts for selecting a virtual
object: real hand, virtual offset cursor, and virtual offset hand. Our
results show that the error rate is highest for the real hand condition
and less for the virtual offset-based techniques. However, our results
indicate that selections with the real hand resulted in the highest
effective throughput of all conditions. This suggests that virtual
offset-based techniques do not improve overall performance.
Index Terms: H.5.2 [Information Interfaces and Presentation]:
User Interfaces—Input Devices and Strategies, Evaluation / Methodology.
1 INTRODUCTION
Stereoscopic display poses challenges to the design of user interfaces
for semi-immersive virtual environments (VEs). Standard
(multi-)touch technologies can provide intuitive and natural interaction
with objects displayed with zero parallax on a display surface.
But, sensing human gestures and postures in “mid-air” above
the surface introduces challenges to the design of high-performance
interaction techniques for selection and manipulation [1, 16]. Both
the increase in the degrees-of-freedom that have to be controlled
as well as the absence of passive haptic feedback and resulting interpenetration
and occlusion issues can reduce performance when
touching virtual objects displayed with negative parallax, i. e., in
front of a display surface [1]. Objects displayed behind the surface
with positive parallax cannot be reached by direct interaction.
Using direct input in such setups suffers from inherent visual
conflicts [15]. In particular, when a user tries to touch a virtual object
that is displayed with negative parallax, either the virtual object
or the user’s finger will appear blurred. While visual distance cues
from the convergence angle to the virtual object and to the user’s
finger may indicate that they are at the same position, the focus
distance is usually adjusted to the user’s real finger, with the more
distant display surface being out of focus. This results in the finger
appearing sharp, whereas the virtual object appears blurred (see
Figure 1(a)). However, such relative differences in blur between objects
are used by the perceptual system to judge interrelations and
relative distances between objects [5]. This visual problem thus results
in an ambiguity when trying to select a virtual object using
a finger or input device and may significantly impact user performance.
A possible solution to these visual conflicts comes from research
on direct interaction versus interaction at a distance [2, 11, 12]. A
simple solution is to place a virtual cursor with a short offset next to
the user’s tracked finger and then to use this offset cursor to select
other virtual objects. Since both the offset cursor and the virtual objects
are displayed stereoscopically, there is then no more mismatch
in blur between real and virtual object (see Figure 1(b)). While this
reduces visual conflicts, it is not clear whether this results in improved
overall selection task performance. In particular, decoupling
the motor and visual space during natural hand interaction may degrade
performance due to the kinematics of point and grasp gestures
in 3D space and the underlying cognitive functions [9, 17]. Moreover,
using a virtual offset cursor does not eliminate all visual conflicts
in stereoscopic display environments. With increased parallax
and distance from the display surface, the vergence-accommodation
conflict increases, which has been found to affect size and distance
judgments, as well as judgments of interrelations between
displayed virtual objects [8]. To account for this limitation, multiple
approaches have been proposed to better support spatial perception
in stereoscopic setups. For example, research suggests that
spatial judgments benefit from familiar size cues being provided to
the user, such as using a virtual hand for interaction with virtual
objects [15, 18] (see Figure 1©).
Mid-air direct-touch interaction in stereoscopic display environments
poses challenges to the design of 3D user interfaces. Not
only is passive haptic feedback usually absent when selecting a virtual
object displayed with positive or negative parallax relative to
a display surface, but such setups also suffer from inherent visual
conflicts, such as vergence/accommodation mismatches and double
vision. In particular, if the user tries to select a virtual object with
a finger or input device, either the virtual object or the user’s finger
will appear blurred, resulting in an ambiguity for selections that
may significantly impact the user’s performance.
In this paper we evaluate the effect of visual conflicts for mid-air
3D selection performance within arm’s reach on a stereoscopic table
with a Fitts’ Law experiment. We compare three different techniques
with different levels of visual conflicts for selecting a virtual
object: real hand, virtual offset cursor, and virtual offset hand. Our
results show that the error rate is highest for the real hand condition
and less for the virtual offset-based techniques. However, our results
indicate that selections with the real hand resulted in the highest
effective throughput of all conditions. This suggests that virtual
offset-based techniques do not improve overall performance.
Index Terms: H.5.2 [Information Interfaces and Presentation]:
User Interfaces—Input Devices and Strategies, Evaluation / Methodology.
1 INTRODUCTION
Stereoscopic display poses challenges to the design of user interfaces
for semi-immersive virtual environments (VEs). Standard
(multi-)touch technologies can provide intuitive and natural interaction
with objects displayed with zero parallax on a display surface.
But, sensing human gestures and postures in “mid-air” above
the surface introduces challenges to the design of high-performance
interaction techniques for selection and manipulation [1, 16]. Both
the increase in the degrees-of-freedom that have to be controlled
as well as the absence of passive haptic feedback and resulting interpenetration
and occlusion issues can reduce performance when
touching virtual objects displayed with negative parallax, i. e., in
front of a display surface [1]. Objects displayed behind the surface
with positive parallax cannot be reached by direct interaction.
Using direct input in such setups suffers from inherent visual
conflicts [15]. In particular, when a user tries to touch a virtual object
that is displayed with negative parallax, either the virtual object
or the user’s finger will appear blurred. While visual distance cues
from the convergence angle to the virtual object and to the user’s
finger may indicate that they are at the same position, the focus
distance is usually adjusted to the user’s real finger, with the more
distant display surface being out of focus. This results in the finger
appearing sharp, whereas the virtual object appears blurred (see
Figure 1(a)). However, such relative differences in blur between objects
are used by the perceptual system to judge interrelations and
relative distances between objects [5]. This visual problem thus results
in an ambiguity when trying to select a virtual object using
a finger or input device and may significantly impact user performance.
A possible solution to these visual conflicts comes from research
on direct interaction versus interaction at a distance [2, 11, 12]. A
simple solution is to place a virtual cursor with a short offset next to
the user’s tracked finger and then to use this offset cursor to select
other virtual objects. Since both the offset cursor and the virtual objects
are displayed stereoscopically, there is then no more mismatch
in blur between real and virtual object (see Figure 1(b)). While this
reduces visual conflicts, it is not clear whether this results in improved
overall selection task performance. In particular, decoupling
the motor and visual space during natural hand interaction may degrade
performance due to the kinematics of point and grasp gestures
in 3D space and the underlying cognitive functions [9, 17]. Moreover,
using a virtual offset cursor does not eliminate all visual conflicts
in stereoscopic display environments. With increased parallax
and distance from the display surface, the vergence-accommodation
conflict increases, which has been found to affect size and distance
judgments, as well as judgments of interrelations between
displayed virtual objects [8]. To account for this limitation, multiple
approaches have been proposed to better support spatial perception
in stereoscopic setups. For example, research suggests that
spatial judgments benefit from familiar size cues being provided to
the user, such as using a virtual hand for interaction with virtual
objects [15, 18] (see Figure 1©).