I have always been bad at calculus. I think it all started with I was locked into a good grade in college and I stopped going to class. When my next calc class started I had a trash teacher and have been digging myself out ever since. Because of this, I haven’t been able to figure out what the heck is going on with gradients while working with TFv2.

I would see that we would have 4 x 4 equal to 16 but when we did a gradient on that same thing we would get 8. So, here is my attempt to write this out and explain what is going on. I also have a slightly longer Colab Notebook on GitHub.

First, we have the equation y = 4 and z = y^2. We then try and find the derivate \frac{d_z}{d_y}.

If you know calculus, which it appears EVERY web site I go to assumes you do, you can see that to find this derivative you use the power rule. This states that you can convert x^n to nx^{n-1}

Using that, we can see y^2 becomes 2y^1 and since y=4 we get 8

Now, to the second example that uses a cube. x = 2, y = x^3, and z is \frac{d_y}{d_x}. This then gives use 3x^2 and using $x = 2$ we get 8

Hopefully, this clears up what is going one. If not, you can just call me a dummy like I am sure everyone else already does when I try and do calculus.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s