- Introduction to neural networks
- Supervised and Unsupervised Learning
- Feed-forward Backpropagation Neural Networks
- Backpropagation Algorithm Overview
- Learning Rate
- Momentum and Jitter
- Activation Functions
- Kohonen Self-Organizing Maps
- Neighborhood Functions
- Lattice Topology
- Kohonen Layer Shape
- Neural Network Design
- Applications of neural networks
- Setting up the development environment
- NeuronDotNet Class Overview
- Some useful code snippets
- NeuronDotNet Sample Applications
Kohonen Self-Organizing Maps
Kohonen SOMs (Self-Organizing Maps) are unsupervised learning neural networks widely used to reduce dimensionality of input space preserving its topological structure. A typical Kohonen SOM architecture is shown below. It consists of an input layer connected to a output layer (Two-dimensional Kohonen layer) via a Kohonen Connector consisting of Kohonen Synapses. Each neuron in a Kohonen Layer is associated with a unique set of co-ordinates in two-dimensional space, and hence is referred to as a Position Neuron. The input layer with 'n' input neurons is fed with n-dimensional input data one by one. The output layer organizes itself to represent the inputs. Hence the name 'self-organizing map'.
During the training phase, a SOM builds a representation of training samples. The trained network can be used to map any input vector onto two-dimensional space.
The objective of SOM training is to ensure that different parts of the network respond similarly to similar input vectors. So, the training mainly involves analysing the behaviour of the network for a training sample and adjusting the weights of synapses to ensure that the network exhibits a similar behaviour for a similar input. The training procedure involves following steps.
- Initialize the weights to small random values
- Choose a random training sample, assign the input vector to the input neurons and run the network.
- The output of a neuron in output layer represents the similarity between its weight vector (source synapse weights) and the input vector. The output neuron which has the highest output value is declared as the winner neuron for current input.
- Calculate the distance of each output neuron from the winner using a Neighborhood function
- Update the weights of synapses using following formulae. (This adjusts the weights corresponding to winner and its neighbors such that, a neuron close to the current winner wins for a similar input)
Similarity = Difference between source neurons output and the synapse weight
Weight Change = (Learning Rate) * (Neighborhood value of target neuron) * Similarity
- Similarly, train all samples in a random order. This completes one training cycle (or Training Epoch)
- Repeat the steps to complete the specified number of training epochs.
The trained SOM maps any input vector to a winner neuron, which can be interpreted as the position of vector in two-dimensional space.