18-01-2013, 02:23 PM
REAL TIME HAND GESTURE RECOGNITION SYSTEM FOR DYNAMIC APPLICATIONS
REAL TIME HAND GESTURE RECOGNITION.pdf (Size: 579.39 KB / Downloads: 98)
ABSTRACT
Virtual environments have always been considered as a means for more visceral and efficient human
computer interaction by a diversified range of applications. The spectrum of applications includes analysis
of complex scientific data, medical training, military simulation, phobia therapy and virtual prototyping.
Evolution of ubiquitous computing, current user interaction approaches with keyboard, mouse and pen are
not sufficient for the still widening spectrum of Human computer interaction. Gloves and sensor based
trackers are unwieldy, constraining and uncomfortable to use. Due to the limitation of these devices the
useable command set based diligences is also limited. Direct use of hands as an input device is an
innovative method for providing natural Human Computer Interaction which has its inheritance from textbased
interfaces through 2D graphical-based interfaces, multimedia-supported interfaces, to full-fledged
multi-participant Virtual Environment (VE) systems. Conceiving a future era of human-computer
interaction with the implementations of 3D application where the user may be able to move and rotate
objects simply by moving and rotating his hand - all without help of any input device.
The research effort centralizes on the efforts of implementing an application that employs computer vision
algorithms and gesture recognition techniques which in turn results in developing a low cost interface
device for interacting with objects in virtual environment using hand gestures. The prototype architecture
of the application comprises of a central computational module that applies the camshift technique for
tracking of hands and its gestures. Haar like technique has been utilized as a classifier that is creditworthy
for locating hand position and classifying gesture. The patterning of gestures has been done for recognition
by mapping the number of defects that is formed in the hand with the assigned gestures. The virtual objects
are produced using Open GL library. This hand gesture recognition technique aims to substitute the use of
mouse for interaction with the virtual objects. This will be useful to promote controlling applications like
virtual games, browsing images etc in virtual environment using hand gestures.
INTRODUCTION
The impendent of virtual environments brings in a whole new set of problems for user interfaces.
The unveiling of 3D objects and worlds in which the user is engrossed allows such people as
International Journal of UbiComp (IJU), Vol.3, No.1, January 2012
scientists, engineers, doctors and architects to envision composite structures and systems with
eminent degrees of quality and naturalism. Shutter glasses furnish a stereo or 3D view of the
scene, which is no longer confined to a desktop monitor, but may be a large table, projection
screen or room. The limiting component in these systems currently is the fundamental interaction.
Virtual environments seek to produce a world where the interaction experiences are real. Current
mechanical, acoustic and magnetic input devices track the user and provide control of movement,
selection and manipulation of objects in virtual scenes. Several tools are purported and used so far
to make such interaction more and more prompt and effortless. Touch screens are the most widely
used example: though the ramification of the underlying system is hidden from the user, and
makes it possible for a user to point to the choices as he could do in real life. The cost associated
to it is the major limitations of the aforesaid technology other limitations may be size, requirement
of a physical location, and other intrinsic limitation to 2D. Other more innovative devices
proposed for virtual reality include gloves or wearable tools such as mechanical sensors, actuators
and micro cameras [1]. They are capable to handle 3D worlds, making it natural and realistic, and
also provide in some implementations tactile sensations. Regrettably, their cost is usually very
high, and thus the user acceptance confined, hence making them more desirable for professional
applications such as a flight simulator or remote surgery equipment .However these interactions
are often limited and non rational, while the devices are awkward, unmanageable and prone to
distortion from the physical environment. We are interested in formulating an alternative, natural
interface that more intimately models the way we interact with the real world. The user should be
able to reach out, grab, point and move 3D objects just as we do with real objects.
These challenges open a new direction for human-computer interaction which combined with
computer vision techniques and it is possible to build an advanced input devices. The computer
vision devices can be implemented and upgrade to the new input devices in the future. It gives
the input command to the computer rather than just a function of taking photo or record video.
We can do more implementation to transform the computer vision devices to become an input
command device to reach the function as keyboard or mouse. One of the ways to give signal to
computer vision devices is by using hand gesture. More specifically hand gesture is used as the
signal or input modality to the computer. Certain signal can be recognized by computer as an
input of what computer should do. These will benefits the entire user without using a direct
device and can do what they want as long as the computer vision device can sense it. These make
computer user easier than using the keyboard or mouse. The future computer or laptop may
eliminate the use of keyboard and mouse by substituting with a vision-based interpretation
devices.
Interaction between humans comes from different sensory modes like gesture, speech, facial and
body expressions [2]. The main advantage of using hand gestures is to interact with computer as a
non-contact human computer input modality. The state of art of human computer interaction
presents the facts that for controlling the computer processes gestures of various types of hand
movements have been used .The present research effort defines an environment where a number
of challenges have been considered for obtaining the hand gesture recognition techniques in the
virtual environment. Being an interesting part of the Human computer interaction hand gesture
recognition needs to be robust for real life applications, but complex structure of human hand
presents a series of challenges for being tracked and interpreted. Other than the gesture
complexities like variability and flexibility of structure of hand other challenges include the shape
of gestures, real time application issues, presence of background noise and variations in
illumination conditions. The specifications also involve accuracy of detection and recognition for
International Journal of UbiComp (IJU), Vol.3, No.1, January 2012
real life applications [3].
The present research effort has a goal of developing an application using vision based hand
gestures for manipulation of objects in virtual environment. Our application presents a more
effective and user friendly methods of human computer interaction intelligently with the usage of
hand gestures. Functions of mouse like controlling of movement of virtual object have been
replaced by hand gestures. The complexity involved is with the detection and recognition phases
of the simulated virtual application. The challenges encountered are noisy environment which
creates a big impingement on the detection and recognition performance of human hand gestures.
The application has been designed to be cost effective and uses low cost input tools like webcam
for capturing hand as input. Manipulation of virtual objects has been done through modeling of
some predefined command based hand gestures.
STATE OF ART
In earlier days hand gesture detection was done using mechanical devices to obtain information
of the hand gesture [4]. One of the most widely used and accepted examples for hand gestures
recognition is data glove. Evolution of computer hardware improved a lot of in present scenario
this also effects the performance of computing. Enhancements of gesture recognition has replaced
the role of data gloves to non wearable devices due to its naturalness without using any device
this is quite user friendly in human computer interaction. One of the major drawbacks of data
glove is that it is cumbersome with the limitation of hand movement.
Figure 1. Three common stages of gesture recognition systems
Also vision is one of the major six physical senses that computer must be instantiated perceptibly
when communicated to humans [1]. So vision based approaches are preferred more than wearable
devices in hand gesture recognition. Generally there are three stages in most of the gesture
recognition systems. The three stages may be enumerated as image pre –processing tracking and
recognition stage [16] as shown in Figure 1.In tracking, there are several researchers who have
done the similar research like Viola-Jones based cascade classifier, commonly used for face
International Journal of UbiComp (IJU), Vol.3, No.1, January 2012
tracking in rapidly image processing [5]. Cascade classifiers are currently considered more robust
pattern detection against the noises and lighting conditions as well [6].
For tracking Viola-Jones and several other researchers have developed algorithms used for face
tracking in rapid image processing like HAAR cascade classifier. This is presently one of the
robust detection techniques under different constraints like noise [6].Gesture as input of human
computer interaction based applications is an emerging field in which many researchers have
worked and proposed different practical techniques. Jain [7] implemented a vision based hand
gesture pose estimation based application for mobile devices. Pavlovic et al. [8] accomplished in
their work that the gestures of users must be explained logically for developing a good human
computer interaction based system .Though the present technologies of gesture recognition are
not feasible in providing the logical explanations to the human gestures. Some of the major
challenges evolved in due course of time are the complexness and robustness of human gestures.
Another hand gesture recognition method based on input-output Hidden Markov Models of
tracking skin color blobs was proposed by Marcel et al. [9]. Controlling VLC media player using
hand gesture recognition is done in real time environment using vision based techniques [10]. The
sign language tutoring tool studied by Aran et al. [11] which their research designed to teaching
the fundamental of the sign language in interactive way.