Sigmoid function — ‘S’ shaped function

Step by step implementation with its derivative

3 min readDec 1, 2021

With this post, we are starting the third Chapter — Activation functions and their derivatives. How they are used in Deep Learning will be discussed later. The most important post in this chapter is the last one where we will talk about the Softmax activation function and its Jacobian which very few people talk about. These posts are very short. So, let us begin with the first Activation function, i.e., the Sigmoid function.

You can download the Jupyter Notebook from here.

Back to the previous post

Back to the first post

Note — In all of the posts of Chapter 3, the input array is of shape (-1, 1) but you can use them in any shape whether it is (-1, 1), (-1,), or (1, -1)

3.1 What is the Sigmoid activation function and its derivative?

This is the definition of the Sigmoid function.

And it is very easy to find the derivative of the Sigmoid function.

This is the graph for the Sigmoid function and its derivative. Here we can see that the output range of the Sigmoid function is (0, 1)

Sigmoid function and its derivative graph

We can easily implement the Sigmoid function in Python.

import numpy as np                             # importing NumPy
np.random.seed(42)def sig(x):                                    # Sigmoid
    return 1/(1 + np.exp(-x)) def sig_dash(x):                               # Sigmoid Derivative
    return sig(x) * (1 - sig(x))

Defining Sigmoid function and its derivative

Let us have a look at an example.

x = np.array([[0.1], [0.2], [0.3], [0.4], [0.5]])
print(x)sig(x)sig_dash(x)

Example for the Sigmoid function and its derivative

Note — Here the functions ‘sig’ and ‘sig_dash’ are broadcasted to every scalar element in ‘x’.

We will talk more about how to interpret the Sigmoid function when we will talk about the Binary Cross-entropy loss function.

I hope now you understand how to implement the Sigmoid function and its derivative.

Watch the video on youtube and subscribe to the channel for videos and posts like this.
Every slide is 3 seconds long and without sound. You may pause the video whenever you like.
You may put on some music too if you like.

The video is basically everything in the post only in slides.