22-02-2013, 11:42 AM
The Camera Mouse: Visual Tracking of Body Features to Provide Computer Access for People With Severe Disabilities
The Camera Mouse.doc (Size: 679 KB / Downloads: 17)
Abstract
The “Camera Mouse” system has been developed to provide computer access for people with severe disabilities. The system tracks the computer user’s movements with a video camera and translates them into the movements of the mouse pointer on the screen. Body features such as the tip of the user’s nose or finger can be tracked. The visual tracking algorithm is based on cropping an online template of the tracked feature from the current image frame and testing where this template correlates in the subsequent frame. The location of the highest correlation is interpreted as the new location of the feature in the subsequent frame. Various body features are examined for tracking robustness and user convenience. A group of 20 people without disabilities tested the Camera Mouse and quickly learned how to use it to spell out messages or play games. Twelve people with severe cerebral palsy or traumatic brain injury have tried the system, nine of whom have shown success. They interacted with their environment by spelling out messages and exploring the Internet.
INTRODUCTION
EOPLE who are quadriplegic and nonverbal—for ex- ample, from cerebral palsy, traumatic brain injury, or stroke—often have great difficulties communicating their desires, thoughts, and needs. They use their limited voluntary motions to communicate with family, friends, and other care providers. Some people can move their heads. Some can blink or wink voluntarily. Some can move their eyes or tongue. Assistive technology devices have been developed to help them use their voluntary movements to control computers. People with disabilities can then communicate through spelling or expression-building programs. This allows users to exhibit their thoughts, emotions, and intellectual potential. Along with the increased ability to communicate, users with severe disabilities can benefit from computer access in many other ways.
SYSTEM OVERVIEW
The Camera Mouse system currently involves two computers that are linked together—a “vision computer” and a “user com- puter.” A schematic plan of the system is shown in Fig. 1. The vi- sion computer executes the visual tracking algorithm and sends the position of the tracked feature to the user computer. The user computer interprets the received signals and runs any applica- tion software the user wishes to use. The functionalities of the two computers could be integrated into one computer, but the current setup assures sufficient processing power for the visual tracking and allows a supervisor to monitor the tracking perfor- mance without interrupting the user’s actions.
Vision Computer
The vision computer receives and displays a live video of the user sitting in front of the user computer. The video is taken by a camera that is mounted above or below the monitor of the user computer. Watching this video, the user or an attending care provider clicks with the vision computer’s mouse on the feature in the image to be tracked, perhaps the tip of the user’s nose. The camera’s remote control can be used to initially adjust the pan and tilt angles of the camera and its zoom so that the desired body feature is centered in the image. The vision system deter- mines the coordinates of the selected feature in the initial image and then computes them automatically in subsequent images.
TRACKING ALGORITHM
When the user initially clicks on the feature to be tracked, a square is drawn around the feature and the subimage within this square is cropped out of the image frame. The cropped subimage is used as a “template” to determine the position of the feature in the next image frame. To find this position, the tracking al- gorithm uses the template to search for the feature in a “search window” that is centered at the position of the feature in the pre- vious frame. The template is shifted through this search window and correlated with the underlying subimages. The window is defined to contain the centers of all subimages tested. Fig. 4 il- lustrates template and search window positions.
Thumb Tracking
To test other body features, not just facial features, the thumb was selected. Although it was tracked successfully, as shown in sequence E in Fig. 7, it has two main flaws as a tracking point. First, the camera has difficulties in focusing on it. As can be seen in sequence E, the thumb takes up such a small portion of the screen that the camera’s autofocus mechanism focuses on the objects in the background and not the thumb. This distorts the outline of the thumb and makes it difficult to track. Thus, if the thumb is used as a tracking point, it should be held close to the body or some other object in the background, so that the camera is able to focus on the thumb correctly. Another problem in using the thumb is its small surface area, which can move out of the search window easily. If this occurs, the tracking program will lose the thumb entirely and begin tracking a new point in the background. This means that if the thumb is used, slow move- ments are necessary.
Dark Lighting
A set of test images was taken in a room with no natural or artificial light except for what was produced by the computer monitor. As can be seen in sequence F in Fig. 7, the Camera Mouse was able to track the person’s nose in these lighting con- ditions.
CONCLUSION
The experiences with the Camera Mouse system are very en- couraging. They show that the Camera Mouse can successfully provide computer access for people with severe disabilities. It is a user-friendly communication device that is especially suitable for children. The system tracks many body features and does not have any user-borne accessories, so it is easily adaptable to serve the special needs of people with various disabilities. To meet the current demand, additional Camera Mouse systems are being installed. A single-computer version of the system is being developed. Future work will incorporate a detection com- ponent into the visual tracking algorithm.4