Description
The Artificial Neural Networks are
inspired from biological neurons in our brains.
The neurons in are brains transmit information if the electrical signal value is above a certain
threshold value.
Similarly, in neural networks, input values are transmitted through layers of neurons using the weights
of the neurons and an output value is produced.
It is a supervised machine learning
technique which means that we know the exact value of the inputs.
The produced output value is compared the actual value of the input and weights in the nodes are updated
accordingly.
This procedure is repeated until there is no need to update weight in the network, which means produced
output is equal to the exact value of the input.
Input Layer
A neural network contains neurons and connection between neurons.
Layers are the combination of neurons and they have different meanings.
There are three
types of layers; input layer, hidden layers and output layers.
Input layer is used to feed the input, for example, if your input consists of 3 numbers, your input layer
would have 3 nodes.
Hidden Layers
The hidden layers are the layers between the input and output layers.
It is possible to have more than 1 hidden layer.
If there are 2 or more hidden layers in a neural network, it is call as Deep Neural Network.
The increase in the size of the hidden layer amount results in more computation cost.
But it also enables neural networks to solve more complex problems.
Output Layer
The output layer is the one producing the final result of the neural network model.
All neural networks has one output layer.
The information is transmitted through the hidden layers.
Even though there may be more than 1 hidden layer, the output layer receives inputs only from the last
hidden layer.
Neuron
Neural networks' basic unit is called a neuron or a Perceptron
Each neuron has its own data(v).
The connection (edge) between layers has their own value (weight).
The multiplication of neuron's value and the weight value of the edge to the successor neuron.
Activation Function
Activation functions can be defined as gates
between neurons.
Activation functions defines which information will be transmitted.
The summation of the multiplication of each neurons data and edge's weight is used as an input to the
activation function.
If result is above to a certain threshold value is transmitted.
Most commanly used activation functions are:
Information Flow
In the figure, there are two inputs (x1, x2) and bias (b) going into the neuron.
Inputs are multiplied with related weight.
The addition of bias value to this summation results in the output (y).
For example let say we use step
function as the activation function.
According to the step function;
- if input value is greater than or equal to 0, then output value will be 1.
- if input value is smaller than 0, then output value will be 0.
Information is transmitted by repeating this process from input layer to the output layer.