Download Artificial Neural Networks

SELF ORGANISING
NETWORKS/MAPS (SOM)
AND
NEURAL NETWORK APPLICATIONS
Outcomes
 Look at the theory of self-organisation.
 Other self-organising networks
 Look at examples of neural network
applications
Four requirements for SOM
Weights in neuron must represent a class of
pattern
 one neuron, one class
Four requirements for SOM
Inputs pattern presented to all neurons and
each produces an output.
 Output: measure of the match between input
pattern and pattern stored by neuron.
Four requirements
A competitive learning strategy selects neuron
with largest response.
Four requirements
A method of reinforcing the largest response.
Architecture
 The Kohonen network (named after Teuvo
Kohonen from Finland) is a self-organising
network
 Neurons are usually arranged on a 2dimensional grid
 Inputs are sent to all neurons
 There are no connections between neurons
Architecture
X
Kohonen network
Theory
 For a neuron output (j) is a weighted
some:
 Where x is the input, w is the weights,
net is the output of the neuron
Four requirement-Kohonen
networks
 True
 Euclidean distance and weighted sum
 Winner takes all
 Learning rule of Kohonen learning
Output value
 The output of each neuron is the weighted
sum
 There is no threshold or bias
 Input values and weights are normalized
“Winner takes all”
 Initially the weights in each neuron are
random
 Input values are sent to all the neurons
 The outputs of each neuron are compared
 The “winner” is the neuron with the largest
output value
Training
 Having found the winner, the weights of the
winning neuron are adjusted
 Weights of neurons in a surrounding
neighbourhood are also adjusted
Neighbourhood
X
neighbourhood
Kohonen network
Training
 As training progresses the neighbourhood
gets smaller
 Weights are adjusted according to the
following formula:
Weight adjustment
 The learning coefficient (alpha) starts with a
value of 1 and gradually reduces to 0
 This has the effect of making big changes to the
weights initially, but no changes at the end
 The weights are adjusted so that they more
closely resemble the input patterns
Example
 A Kohonen network receives the input
pattern 0.6 0.6 0.6.
 Two neurons in the network have
weights 0.5 0.3 0.8 and -0.6 –0.5 0.6.
 Which neuron will have its weights
adjusted and what will the new values
of the weights be if the learning
coefficient is 0.4?
Answer
The weighted sums are 0.96 and –0.3 so the first neuron wins.
The weights become:
w1 = 0.5 + 0.4 *(0.6 – 0.5)
w1 = 0.5 + 0.4 * 0.1 = 0.5 + 0.04 = 0.54
w2 = 0.3 + 0.4 *(0.6 – 0.3)
w2 = 0.3 + 0.4 * 0.3 = 0.3 + 0.12 = 0.42
w2 = 0.8 + 0.4 *(0.6 – 0.8)
w2 = 0.8 - 0.4 * 0.2 = 0.8 - 0.08 = 0.72
Summary
 The Kohonen network is self-organising
 It uses unsupervised training
 All the neurons are connected to the input
 A winner takes all mechanism determines which
neuron gets its weights adjusted
 Neurons in a neighbourhood also get adjusted
Demonstration
 A demonstration of a Kohonen network
learning has been taken from the following
websites:
 http://www.patol.com/java/TSP/index.html
 http://www.samhill.co.uk/kohonen/index.htm
Applications of Neural Networks
ARTIFICIAL INTELLIGENCE
TECHNIQUES
Example Applications
 Analysis of data
 Classifying in EEG
 Pattern recognition in ECG
 EMG disease detection.
Gueli N et al (2005) The influence of lifestyle on cardiovascular
risk factors analysis using a neural network Archives of
Gerontology and Geriatrics 40 157–172
 To produce a model of risk facts in heart
disease.
 MLP used
 The accuracy was relatively good for
chlorestremia and triglyceremdia:
 Training phase around 99%
 Testing phase around 93%
 Not so good for HDL
Subasi A (in press) Automatic recognition of alertness level from
EEG by using neural network and wavelet coefficients Expert Systems
with Applications xx (2004) 1–11
 Electroencephalography (EEG)
 Recordings of electrical activity from the brain.
 Classifying operation
 Awake
 Drowsy
 Sleep
 MLP
 15-23-3
 Hidden layer – log-tanh function
 Output layer – log-sigmoid function
 Input is normalise to be within the range 0 to
1.
 Accuracy
 95%+/-3% alert
 93%+/-4% drowsy
 92+/-5% sleep
 Feature were extracted and form the input to
the network, from wavelets.
Karsten Sternickel (2002) Automatic pattern recognition in
ECG time series Computer Methods and Programs in
Biomedicine 68 109–115
 ECG – electrocardiographs – electrical signals
from the heart.
 Wavelets again.
 Classification of patterns
 Patterns were spotted
Abel et al (1996) Neural network analysis of the EMG
interference pattern Med. Eng. Phys. Vol. 18, No. 1.
pp. 12-l 7
 EMG – Electromyography – muscle activity.
 Interference patterns are signals produce
from various parts of a muscle- hard to see
features.
 Applied neural network to EMG interference
patterns.
 Classifying
 Nerve disease
 Muscle disease
 Controls
 Applied various different ways of presenting
the pattern to the ANN.
 Good for less serve cases, serve cases can
often be see by the clinician.
Example Applications
 Wave prediction
 Controlling a vehicle
 Condition monitoring
Wave prediction
 Raoa S, Mandal S(2005) Hindcasting of storm
waves using neural networks Ocean Engineering
32 (2005) 667–684
 MLP used to predict storm waves.
 2:2:2 network
 Good correlation between ANN model and
another model
van de Ven P, Flanagan C, Toal D (in press)
Neural network control of underwater vehicles
Engineering Applications of Artificial
Intelligence
 Semiautomous vehicle
 Control using ANN
 ANN replaces a mathematical model of the
system.
Silva et al (2000) THE ADAPTABILITY OF A TOOL WEAR
MONITORING SYSTEM UNDER CHANGING CUTTING CONDITIONS
Mechanical Systems and Signal Processing (2000)
14(2), 287-298
 Modelling tool wear
 Combines ANN with other AI (Expert
systems)
 Self organising Maps (SOM) and ART2
investigated
 SOM better for extracting the required
information.
Examples to try yourself
 A.1 Number recognition (ONR)
 http://www.generation5.org/jdk/demos.asp#
neuralNetworks
 Details:
http://www.generation5.org/content/2004/si
mple_ocr.asp
 B.1 Kohonen Self Organising Example 1
 http://www.generation5.org/jdk/demos.asp#
neuralNetworks
 B.2 Kohonen 3D travelling salesman problem
 http://fbim.fhregensburg.de/~saj39122/jfroehl/diplom/eindex.html