25-08-2012, 02:29 PM
BAREHAND HUMAN-COMPUTER INREACTION
bare hand.docx (Size: 389.43 KB / Downloads: 34)
ABSTRACT:
This paper deals with techniques for barehanded interaction between human and computer. Barehanded means that no device and no wires are attached to the user, who controls the Computer directly with the movements of his/her hand. This approach is centered on the needs of the user. Therefore requirements for real-time barehanded interaction are defined and derived from application scenarios and usability considerations. Based on those requirements, which a finger-finding and hand-posture recognition algorithm is developed and evaluated. To
demonstrate the strength of the algorithm, three sample applications are built. Finger tracking
and hand posture recognition are used to paint virtually onto the wall, to control a presentation
with hand postures, and to move virtual items on the wall during a brainstorming session. The
paper is concluded with user tests, which were conducted to prove the usability of bare-hand
human computer interaction.
INTRODUCTION
For a long time research on human-computer interaction has been restricted to techniques based on the use of a graphic display, a keyboard and a mouse. Recently this paradigm has changed. Techniques such as vision, sound, speech recognition, projective displays and context-aware devices allow for a much richer, multimodal interaction between man and machine. Today there are many different devices available for hand-based human-computer interaction. Some examples are keyboard, mouse, track-ball, track-pad, joystick, electronic pens and remote controls. More sophisticated examples include cyber-gloves, 3Dmice (e.g. Labtec’s Space ball)
and magnetic tracking devices (e.g. Polhemus’ Isotrack). But despite the variety of new devices,
human-computer interaction still differs in many ways from human-to-human interaction.Natural interaction between humans does not involve devices because we have the ability to sense our environment with eyes and ears. In principle, the computer should be able to imitate those abilities with cameras and microphones. In this paper, we will take a closer look at humancomputer interaction with the bare hand. In this context, “bare” means that no device has to be in contact with the body to interact with the computer. The position of the hand and the fingers will be used to control applications directly. Our approach will be centered on the needs of the user.Requirements derived from usability consideration will guide our implementation, i.e. we will not try to solve general computer vision problems, but rather find specific solutions for a specific scenario. In the next section of the paper, we will describe applications, where bare-hand input is superior to traditional input devices. Based on these application scenarios, we will set up
functional and non-functional requirements for bare-hand human-computer interaction systems.
In the fourth section, there will be an overview about possible approaches and related work to the
problem. Because none of the current hand-finding and -tracking systems sufficiently fulfill our
requirements,
REQUIREMENTS
In this section, we will define requirements for real-time human computer interaction
with bare hands that will guide our implementation and evaluation later on.
Functional Requirements
Functional requirements can be described as the collection of services that are expected from a system. For a software system, these services can cover several layers of abstraction. In our context, only the basic services are of interest. We identify three essential services for vision based Human computer interaction: detection, identification and tracking. We will briefly present the three services and describe how they are used by our envisaged applications.
Detection
Detection determines the presence and position of a class of objects in the scene. A class of objects could be body parts in general, faces, hands or fingers. If the class contains only one
object type, and there is just one object present in the scene at a time, detection suffices to build
simple applications. For example, if we detect the presence of fingertips and we constrain our
application to one fingertip at a time, the detection output can be used to directly control a mouse
pointer position. For more complex applications, such as hand posture recognition and multi handed Interaction, we will need an additional identification and tracking stage.
HAND SEGMENTATION
When processing video images, the basic problem lies in the extraction of information from vast amount of data. The Matrix Meteor frame grabber, for example, captures over 33 megabytes of data per second, which has to be reduced to a simple fingertip position value in fractions of a second. The goal of the segmentation stage is to decrease the amount of image information by selecting areas of interest. Due to processing power constraints, only the most basic calculations are possible during segmentation. Typical hand segmentation techniques are based on stereo information, color, contour detection, connected component analysis and image differencing.
Hand posture Classification
The second part of the algorithm analyzes the relationship between the found fingers. As a first step, a standard connected component analysis algorithm is used to analyze which of the found fingers belong to the same hand. As a by-product, the size of the hand-region is calculated. This can be used to filter out small finger-shaped objects, such as pens. In a next step, the fingers are sorted into the right geometric order (minimum distance between every pair). Afterwards the
directions and positions of fingers relative to each other allow calculating an approximation for
the center of the palm. Fingers can then be classified by their position relative to the palm and
their position to each other.
CONCLUSION
In this paper, we described how a computer can be controlled with the bare hand. We developed a simple but effective finger finding algorithm that runs in real-time at a wide range of light conditions. Other than in previous work, our system does not constrain the hand movement of the user. Also, there is no set-up stage. Any user can simply walk up to the wall and start
interacting with the system. The described user tests show that the organization of projected
items on the wall can be easily accomplished with bare hand interaction. Even though the system
takes more time than its physical counterpart, we think that it is still very useful: many valueadding services, such as printing and storing, can only be realized with the virtual representation Further research will be necessary to find a faster selection mechanism and to improve the segmentation with a projected background under difficult light conditions.