I'm currently learning Convolutional Neural Networks and am stuck on trying to figure out how to compute gradients in a layer that uses transposed convolution. Also, how do I calculate the gradients if I use padding=1 and stride=2?
Thanks to this article "https://hideyukiinada.github.io/cnn_backprop_strides2.html" I was able to figure out how to calculate gradients in normal convolution and all that remains for me is to figure out how to calculate them in transposed convolution.
In a normal convolution network you can easily calculate the derivative of X*K=Y with respect to X or K by writing down the formula of each cell in Y and thus you can easily derive Y with respect to X or K.
derivative in normal convolution:
dL/dX = K (fully convolved 180) with dL/dY
dL/dK = X (convolved) with dL/dY
But in the transposed convolution network it is exactly the opposite. The calculation of the derivative in a normal convolution is very similar to the normal calculation of transposed convolution and in the same idea the derivative of the transposed convolution is similar to the normal convolution between X or K and the output. And of course you can write down the formula of each cell in Y and derive with respect to X or K and see it for yourself.
derivative of transposed convolution:
dL/dX = K (convolved) with Y
dL/dK = X (convolved) with Y
