03-09-2012, 01:06 PM
ANN for Handwritten Digits Recognition
ANN for Handwritten.pdf (Size: 396.32 KB / Downloads: 161)
Abstract
In this paper we present an Aritificial Neural Network to tackle
the recognition of human handwritten digits. The ANN proposed
here is experimented on the well-known MNIST data set. Without any
preprocessing of the data set, our ANN achieves quite low classification
error. Combined with clustering techniques, we can build artificial
intelligence system which can automatically segment individual digit
from images and find its corresponding label.
Introduction
Automatic recognition of human handwritten digits was a mysterious
job to me when I was an undergraduate student. Different
people have very different writing style, even digits of
a same person written in different time are not identical. How
does artificial intelligence deal with the infinite possibility of different
shapes of digits, given only an image? Since now I have
taken the Machine Learning course and acquired knowledge in
this field, I am able to tackle this problem with my own matlab
program.
Methods
The program I implement will mainly focus on identifying 0-9
from segmented pictures of handwritten digits. The input of my
program is a gray level image, the intensity level of which varies
from 0 to 255. For simplicity, input images are pre-treated to
be of certain fixed size, and each input image should contain
only one unknown digit in the middle. These requirements are
not too harsh because they can be achieved using simple image
processing or computer vision techniques. In addition, such
pre-treated image data set are easy to obtain. In my implementation,
the popular MNIST data set ([1]) is a good choice. Each
image in MNIST is already normalized to 28x28 in the above
sense and the data set itself is publicly available. The MNIST
data set is really a huge one: it contains 60000 training samples
and 10000 test samples. And it has become a standard data set
for testing various algorithms.
Discussion
The MNIST data set is a popular data set, on which various
classification algorithms has been tested. The state of art on
this data set is large Convolutional Neural Network ([1]) with
unsupervised pretraining. It achieved an error rate of 0.39% on
test set. However, to make a fair comparison, it’s more beneficial
to compare performances of different algorithms without any
preprocessing on the data set. Table 1 lists the error rates of
several algorithm, applied on the original data set.
Conclusion
In conclusion, I implement a large multilayer artificial neural
network for human handwritten digits. I train the ANN with
cross entropy using error back propagation to obtain optimal
weights values. I also find useful applications for the ANN I
generate. By doing this project, I have practiced using what
I have learned in the Machine Learning course. My program
probably does not beat the state of the art in handwritten digit
recognition. However, I have for the first time observed the practical
problems of using the powerful Artificial Neural Networks,
for example, designing the architecture of an ANN, choosing appropriate
activation functions for each layer, error back propagation,
convergence issues, stopping criteria, generalization ability
. . .This experience will definitely be helpful for my future research.