Updating Parameters

After back propagation, we have gradients for all the weights and biases in our network. Assuming you know what gradient descent is, all you need to do is fix some learning rate and update all the weights in the network.

Final task

• Add one more function to update the parameters in the network.

• You can pass the grads dictionary that has the gradients which are calculated during the back propagation.

• In addition to grads, you can also pass learning rate as the input parameter. This allows you to increase or decrease the learning rate for every few epochs.

If you wish, you can include this updation as a part of back propagate function by making the necessary changes.

Optional

This is a very simple implementation with single Neuron at the output layer, with sigmoid as the activation function in all the layers and parameter updation with vennela gradient descent.

• You can have different activations for different layers

• Use other variations of GD like Momentum, Adam etc.,

• Try using multiple neurons at the end with softmax as the activation function to get the probabilities.

References

• https://medium.com/@udaybhaskarpaila/multilayered-neural-network-from-scratch-using-python-c0719a646855

https://medium.com/@udaybhaskarpaila/multilayered-neural-network-from-scratch-using-python-c0719a646855\medium.com

\

• CS7015 - IIT Madras course.

libraries used to create the content

• https://www.mathcha.io/

Mathcha - Online Math Editor

• https://katex.org/

KaTeX – The fastest math typesetting library for the web