24-07-2014, 04:43 PM
Kazim Sekeroglu
Kazim Sekeroglu.pdf (Size: 214.81 KB / Downloads: 93)
INTRODUCTION
Since the computer technology continues to grow up, the importance of human computer
interaction is enormously increasing. Nowadays most of the mobile devices are using a touch
screen technology. However, this technology is still not cheap enough to be used in desktop
systems. Creating a virtual human computer interaction device such as mouse or keyboard
using a webcam and computer vision techniques can be an alternative way for the touch
screen. In this study, finger tracking based a virtual mouse application has been designed and
implemented using a regular webcam. The motivation was to create an object tracking
application to interact with the computer, and develop a virtual human computer interaction
device.
“Many researchers in the human computer interaction and robotics fields have tried to
control mouse movement using video devices. However, all of them used different methods
to make a clicking event. One approach, by Erdem et al, used fingertip tracking to control the
motion of the mouse. A click of the mouse button was implemented by defining a screen
such that a click occurred when a user’s hand passed over the region [1]. Another approach
was developed by Chu-Feng Lien [2]. He used only the finger-tips to control the mouse
cursor and click. His clicking method was based on image density, and required the user to
hold the mouse cursor on the desired spot for a short period of time. Paul et al, used still
another method to click. They used the motion of the thumb (from a ‘thumbs-up’ position to
a fist) to mark a clicking event thumb. Movement of the hand while making a special hand
sign moved the mouse pointer [3].”
SYSTEM DESCRIPTION
In the object tracking application one of the main problems is object detection. Instead of
finger tips, a color pointer has been used to make the object detection easy and fast. A circle
blue sticker is used as a color pointer in this study. To simulate the click events of the mouse
three fingers with three color pointers has been used.
The basic algorithm is as follows;
∗ Set a pointer in the image
∗ Detect the pointer using the defined color information
∗ Define the region and the center of the pointer and draw a bounding box around it
Find the Center of the Object and Draw a Bounding Box
After removing all the connected components (objects) other than the pointer, using
MATLAB’s “bwlabel” function the pointer can be labeled. In other words the region can be
detected. To get the properties of the region such as center point or bounding box etc.,
MATLAB’s built in “regionprops” function can be used as;
STATS = regionprops(BW, properties)
Where; it measures a set of properties for each connected component (object) in the binary
image, BW. The image BW is a logical array; it can have any dimension. In our application
Mouse Click Events
Mouse click is one of the most challenging parts for the virtual mouse application. As all we
know, the left button of the mouse has a different task for a single or double clicks. To
simulate this feature I have used another pointer. When the second pointer is detected in the
image, left mouse click event has been executed. Depending on the time that the second
pointer is being detected single and the double clicks have been simulated, and the third
pointer is being used to simulate the right click of the mouse.
CONCLUSION AND FUTURE WORKS
In this study, an object tracking based virtual mouse application has been developed and
implemented using a webcam. The system has been implemented in MATLAB environment
using MATLAB Image Processing Toolbox. As an object a blue color sticker is used to make
the detection easy and fast. Object detection and motion tracking worked very well. Using
the pointer moving the cursor and the simulating the mouse click events also worked well.
However, system has some disadvantages such as; being invariant to illumination up to some
scale, and movement of the cursor is very sensitive to motion. Because of this reason, to
control the cursor, pointer cannot be used on the air efficiently.
For the future works, instead of a color pointer directly finger tips can be detected. To detect
the object salient future detection algorithm such as SIFT, SURF, or ORB can be used.
However, this type of complex algorithm might slow down the speed and the system may not
be used in real time.