Updating Parameters
Last updated
Last updated
After back propagation, we have gradients for all the weights and biases in our network. Assuming you know what gradient descent is, all you need to do is fix some learning rate and update all the weights in the network.
Add one more function to update the parameters in the network.
You can pass the grads dictionary that has the gradients which are calculated during the back propagation.
In addition to grads, you can also pass learning rate as the input parameter. This allows you to increase or decrease the learning rate for every few epochs.
If you wish, you can include this updation as a part of back propagate function by making the necessary changes.
This is a very simple implementation with single Neuron at the output layer, with sigmoid as the activation function in all the layers and parameter updation with vennela gradient descent.
You can have different activations for different layers
Use other variations of GD like Momentum, Adam etc.,
Try using multiple neurons at the end with softmax as the activation function to get the probabilities.
CS7015 - IIT Madras course.
