28-12-2012, 02:58 PM
An Accelerometer-Based Digital Pen With a Trajectory Recognition Algorithm for Handwritten Digit and Gesture Recognition
1An Accelerometer-Based Digital.pdf (Size: 871.94 KB / Downloads: 178)
Abstract
This paper presents an accelerometer-based digital
pen for handwritten digit and gesture trajectory recognition applications.
The digital pen consists of a triaxial accelerometer, a
microcontroller, and an RF wireless transmission module for sensing
and collecting accelerations of handwriting and gesture trajectories.
The proposed trajectory recognition algorithm composes of
the procedures of acceleration acquisition, signal preprocessing,
feature generation, feature selection, and feature extraction. The
algorithm is capable of translating time-series acceleration signals
into important feature vectors. Users can use the pen to write digits
or make hand gestures, and the accelerations of hand motions
measured by the accelerometer are wirelessly transmitted to a
computer for online trajectory recognition. The algorithm first
extracts the time- and frequency-domain features from the acceleration
signals and, then, further identifies the most important
features by a hybrid method: kernel-based class separability for
selecting significant features and linear discriminant analysis for
reducing the dimension of features. The reduced features are sent
to a trained probabilistic neural network for recognition. Our
experimental results have successfully validated the effectiveness
of the trajectory recognition algorithm for handwritten digit and
gesture recognition using the proposed digital pen.
INTRODUCTION
EXPLOSIVE growth of miniaturization technologies in
electronic circuits and components has greatly decreased
the dimension and weight of consumer electronic products,
such as smart phones and handheld computers, and thus made
them more handy and convenient. Due to the rapid development
of computer technology, human–computer interaction
(HCI) techniques [1]–[3] have become an indispensable component
in our daily life. Recently, an attractive alternative,
a portable device embedded with inertial sensors, has been
proposed to sense the activities of human and to capture
his/her motion trajectory information from accelerations for
recognizing gestures or handwriting.
RELATED WORK
Recently, some studies have focused on the development of
digital pens for trajectory recognition and HCI applications.
For instance, an alternative method of conventional tablet-based
handwriting recognition has been proposed by Milner [15]. In
his system, two dual-axis accelerometers are mounted on the
side of a pen to generate time-varying x- and y-axis acceleration
for handwriting motion. The author employed an HMM
with a bandpass filtering and a down-sampling procedure for
classification of seven handwritten words. The best recognition
rate is 96.2% when the number of states of the HMMis equal to
60. Oh et al. [16] presented a wandlike input device embedding
a triaxial accelerometer and a triaxial gyroscope for online 3-D
character gesture recognition. Fisher discriminant analysis was
adopted, and different combinations of sensor signals were used
to test the recognition performance of their device. When all six
axes raw signals were used as inputs of the recognition system,
the recognition rate was 93.23%. In addition, they proposed an
ensemble recognizer consisting of three subrecognizers with
the following signals as inputs: acceleration, angular velocity,
and estimated handwriting trajectory. The recognition rate of
the recognizer was 95.04%. Similarly, a gesture recognition
system consisting of a gesture input device, a trajectory estimation
algorithm, and a recognition algorithm in 3-D space
was proposed by Cho et al. [17]. The trajectory estimation
algorithm based on an inertial navigation system was developed
to reconstruct the trajectories of numerical digits and three
hand gestures, and then, a Bayesian network was trained to
recognize the reconstructed trajectories. The average recognition
rate was 99.2%. Zhou et al. [18] proposed a μIMU for
2-D handwriting applications. They extracted the discrete
cosine transform features from x- and y-axis acceleration
signals and one angular velocity and used an unsupervised
self-organizing map to classify 26 English alphabets and ten
numerical digits. The recognition rate of 26 English alphabets
and ten numerical digits achieved 64.38% and 80.8%,
respectively.
TRAJECTORY RECOGNITION ALGORITHM
The block diagram of the proposed trajectory recognition
algorithm consisting of acceleration acquisition, signal preprocessing,
feature generation, feature selection, and feature
extraction is shown in Fig. 3.
In this paper, the motions for recognition include Arabic
numerals and eight hand gestures. The acceleration signals of
the hand motions are measured by a triaxial accelerometer and
then preprocessed by filtering and normalization. Consequently,
the features are extracted from the preprocessed data to represent
the characteristics of different motion signals, and the
feature selection process based on KBCS picks p features out of
the original 24 extracted features. To reduce the computational
load and increase the recognition accuracy of the classifier, we
utilize LDA to reduce the dimension of the selected features.
The reduced feature vectors are fed into a PNN classifier to
recognize the motion to which the feature vector belongs. We
now introduce the detailed procedure of the proposed trajectory
recognition algorithm as follows.
HARDWARE DESIGN OF DIGITAL PEN
Our digital pen consists of a triaxial accelerometer
(LIS3L02AQ3, STMicroelectronics), a microcontroller
(C8051F206 with a 12-b A/D converter), and a wireless
transceiver (nRF2401, Nordic). The triaxial accelerometer
measures the acceleration signals generated by a user’s hand
motions. The microcontroller collects the analog acceleration
signals and converts the signals to digital ones via the A/D
converter. The wireless transceiver transmits the acceleration
signals wirelessly to a personal computer (PC). The dimension
of the pen-type circuit board is 14 cm × 2 cm × 1.5 cm as
shown in Fig. 1.
The LIS3L02AQ3 is a low-cost capacitive micromachined
accelerometer with a temperature compensation function and a
g-select function for a full-scale selection of ±2 g and ±6 g
and is able to measure accelerations over the bandwidth of
1.5 kHz for all axes. The accelerometer’s sensitivity is set
from −2 g to +2 g in this study. The C8051F206 integrates a
high-performance 12-b A/D converter and an optimized signal
cycle 25-MHz 8-b microcontroller unit (MCU) (8051 instruction
set compatible) on a signal chip. The output signals of
the accelerometer are sampled at 100 Hz by the 12-b A/D
converter. Then, all the data sensed by the accelerometer are
transmitted wirelessly to a PC by an RF transceiver at 2.4-GHz
transmission band with 1-Mb/s transmission rate. The overall
power consumption of the digital pen circuit is 30 mA at 3.7 V.
Therefore, if a typical AA battery (2000 mAh at 1.5 V) is used
as the power of the system, the system requires three batteries
simultaneously, and the lifetime is about 67 h. The schematic
diagram of the pen-type portable device is shown in Fig. 2.
Feature Extraction
For pattern recognition problems, LDA [23] is an effective
feature extraction (or dimensionality reduction method) which
uses a linear transformation to transform the original feature
sets into a lower dimensional feature space. The purpose of
LDA is to divide the data distribution in different classes and
minimize the data distribution of the same class in a new space.
First, two scatter matrices, a within-class scatter matrix SW and
a between-class scatter matrix SB.
Classifier Construction
The PNN was first proposed by Specht [24]. With enough
training data, the PNN is guaranteed to converge to a Bayesian
classifier, and thus, it has a great potential for making classification
decisions accurately and providing probability and
reliability measures for each classification. In addition, the
training procedure of the PNN only needs one epoch to adjust
the weights and biases of the network architecture. Therefore,
the most important advantage of using the PNN is its high speed
of learning. Typically, the PNN consists of an input layer, a
pattern layer, a summation layer, and a decision layer as shown
in Fig. 4. The function of the neurons in each layer of the PNN
is defined as follows.
EXPERIMENTAL RESULTS
In this section, the effectiveness of trajectory recognition
algorithm is validated by the following two experiments:
1) handwritten digit recognition and 2) gesture recognition. The
proposed trajectory recognition algorithm consists of the following
procedures: acceleration acquisition, signal preprocessing,
feature generation, feature selection, and feature extraction.
We collected the acceleration signals of the two experiments
from ten subjects (four females, six males; age 24.1 ± 2.13
years old) in a laboratory environment. We used different
combinations of feature selection and extraction methods and
employed PNN to recognize handwritten digits and hand gestures.
In addition, we compared the recognition results of the
PNN trained by the features from different feature engineering
methods with those of feedforward neural networks (FNNs).
Note that we tested several FNN topologies with different
parameter settings. We only reported the best result of FNN in
this paper.
CONCLUSION
This paper has presented a systematic trajectory recognition
algorithm framework that can construct effective classifiers for
acceleration-based handwriting and gesture recognition. The
proposed trajectory recognition algorithm consists of acceleration
acquisition, signal preprocessing, feature generation, feature
selection, and feature extraction.With the reduced features,
a PNN can be quickly trained as an effective classifier. In the
experiments, we used 2-D handwriting digits and 3-D hand
gestures to validate the effectiveness of the proposed device
and algorithm. The overall handwritten digit recognition rate
was 98%, and the gesture recognition rate was also 98.75%.
This result encourages us to further investigate the possibility of
using our digital pen as an effective tool for HCI applications.