Description
The Artificial Neural Networks are inspired from biological neurons in our brains.
The neurons in are brains transmit information if the electrical signal value is above a certain threshold value.
Similarly, in neural networks, input values are transmitted through layers of neurons using the weights of the neurons and an output value is produced.
It is a supervised machine learning technique which means that we know the exact value of the inputs.
The produced output value is compared the actual value of the input and weights in the nodes are updated accordingly.
This procedure is repeated until there is no need to update weight in the network, which means produced output is equal to the exact value of the input.
Input Layer
A neural network contains neurons and connection between neurons.
Layers are the combination of neurons and they have different meanings.
There are three types of layers; input layer, hidden layers and output layers.
Input layer is used to feed the input, for example, if your input consists of 3 numbers, your input layer would have 3 nodes.
Hidden Layers
The hidden layers are the layers between the input and output layers.
It is possible to have more than 1 hidden layer.
If there are 2 or more hidden layers in a neural network, it is call as Deep Neural Network.
The increase in the size of the hidden layer amount results in more computation cost.
But it also enables neural networks to solve more complex problems.
Output Layer
The output layer is the one producing the final result of the neural network model.
All neural networks has one output layer.
The information is transmitted through the hidden layers.
Even though there may be more than 1 hidden layer, the output layer receives inputs only from the last hidden layer.
Neuron
Neural networks' basic unit is called a neuron or a Perceptron
Each neuron has its own data(v).
The connection (edge) between layers has their own value (weight).
The multiplication of neuron's value and the weight value of the edge to the successor neuron.
Activation Function
Activation functions can be defined as gates between neurons.
Activation functions defines which information will be transmitted.
The summation of the multiplication of each neurons data and edge's weight is used as an input to the activation function.
If result is above to a certain threshold value is transmitted.
Most commanly used activation functions are:
Information Flow
In the figure, there are two inputs (x1, x2) and bias (b) going into the neuron.
Inputs are multiplied with related weight.
The addition of bias value to this summation results in the output (y).
For example let say we use step function as the activation function.
According to the step function;
Information is transmitted by repeating this process from input layer to the output layer.