|
Activation function decides whether a neuron should be activated or not by calculating the weighted sum and further adding bias to it. The motive is to introduce non-linearity into the output of a neu
|
| bet | 3/5 | | Sana | 11.01.2024 | | Hajmi | 4,74 Mb. | | #135217 |
Bog'liq Mashinali o\'qitishga kirish 21-ma\'ruza Nosirov KhActivation function decides whether a neuron should be activated or not by calculating the weighted sum and further adding bias to it. The motive is to introduce non-linearity into the output of a neuron. If we do not apply activation function then the output signal would be simply linear function(one-degree polynomial). Now, a linear function is easy to solve but they are limited in their complexity, have less power. Without activation function, our model cannot learn and model complicated data such as images, videos, audio, speech, etc. Activation function why do we need Non-Linearity? Non-Linear functions are those which have a degree more than one and they have a curvature. Now we need a neural network to learn and represent almost anything and any arbitrary complex function that maps an input to output. Neural Network is considered “Universal Function Approximators”. It means they can learn and compute any function at all. Activation function (Types) 1.Threshold Activation Function — (Binary step function)
A Binary step function is a threshold-based activation function. If the input value is above or below a certain threshold, the neuron is activated and sends exactly the same signal to the next layer.
Activation function A = “activated” if Y > threshold
else not or A=1 if y>threshold 0 otherwise.
The problem with this function is for creating a binary classifier ( 1 or 0), but if you want multiple such neurons to be connected to bring in more classes, Class1, Class2, Class3, etc. In this case, all neurons will give 1, so we cannot decide.
Activation function (Types) 2. Sigmoid Activation Function — (Logistic function)
A Sigmoid function is a mathematical function having a characteristic “S”-shaped curve or sigmoid curve which ranges between 0 and 1, therefore it is used for models where we need to predict the probability as an output.
The Sigmoid function is differentiable, means we can find the slope of the curve at any 2 points.
The drawback of the Sigmoid activation function is that it can cause the neural network to get stuck at training time if strong negative input is provided.
|
|
Bosh sahifa
Aloqalar
Bosh sahifa
Activation function decides whether a neuron should be activated or not by calculating the weighted sum and further adding bias to it. The motive is to introduce non-linearity into the output of a neu
|
