20-07-2012, 04:41 PM
artificial neural networks (ANN)
artificial neural networks (ANN).pdf (Size: 529.96 KB / Downloads: 222)
Introduction
The idea of artificial neural networks (ANN) has been firstly proposed many years ago
[1], however the difficulties in “training” of such networks have postponed the time of their
practical use. Only after discovering successful method of training the ANN received a great
deal of attention. The improvement in computer technology also allowed fast progress in
ANN development. At present the ANN are used in numerous scientific, industrial and other
areas, as a tool for analysis, prediction, control, identification, data processing, etc. [2]
The ANN are designed to work similarly to neural tissue in brain, where many
independent neurons (processing units) are interconnected into one big data-processing
network. In the ANN the “neurons” are represented as mathematical functions, which pass the
data between themselves in some defined and organised manner. The way of organisation of
the connections between the “neurons” defines the features of the ANN, and their capability
to perform certain tasks.
Back-propagation
The training procedure modifies only the weights in the whole network – the weights
store all the "wisdom" of any ANN.
In order to perform the back-propagation algorithm (BP) the target values need to be
known. All the activation functions are set before training, and the weights (W) are set to
some small random numbers (for example, in aNETka they are in a range from -0.1 to +0.1).
When the first set of inputs is presented to the ANN the output values (Y) can be calculated.
Those values are different from the ideal (target values, T), hence the errors (d) can be
calculated for each neuron in output layer.
Learning rate
The value of learning rate (LR) from equation (3) determines the convergence time of
the ANN. If the LR is too low, then the learning time can be very long, and in some cases the
learning can get stuck in deep local minimum. On the other hand, if the LR is too large, then
there might be very large oscillations of the output values of the ANN, which can lead to
divergence of the whole ANN. Therefore the LR should be set to some “medium” value.
However, it is very difficult to decide what means “medium” for various ANN. The rule of a
thumb is that the greater the number of neurons the smaller the value of LR. For very small
ANN (e.g. 5 neurons) the LR can be as large as 2, but for large ANN (e.g. 45 neurons) the
“best” LR can be as low as 0.001. The easiest way of determining the maximum LR is simply
to run the ANN – if there are large oscillations or the ANN diverge, then the LR should be
decreased (for example by half).