Questions about the implementation of Grad-CAM++: The second and third order derivatives

[hxngu]

#541 When I try to learn how to get hessian matrix I find the implementation of Grad-CAM++ just used grads_power_2 = grads**2 and grads_power_3 = grads_power_2 * grads to get the second and third order derivatives. Is that correct? The formula in original paper is listed below.

[caihuaiguang]

GradCAM++ approximates second and third-order derivatives using squared and cubed gradients, which holds only when logits are passed through an exponential function (i.e., output = exp(logits) ).

Computing the exact Hessian matrix is both time-consuming and space-consuming. However, obtaining the Hessian-vector product requires only a single backward pass, as demonstrated in ShapleyCAM, which is also merged to this repository.

[hxngu]

Thanks for your reply. I have tried to use the code of ShapleyCAM. However, I find the Hessian-vector product: hvp = torch.autograd.grad( outputs=grads, inputs=activations, grad_outputs=activations, retain_graph=False, allow_unused=True )[0]is None when I try to use the demo(The Usage examples of this library in github homepage), why would this product be None? Looking forward to your reply.

[caihuaiguang]

I believe this issue arises because the default setting is targets = [ClassifierOutputTarget(281)] (i.e., pre-softmax utility function). To avoid get None, you can use targets = [ClassifierOutputReST(281)] instead.

This is because for most CNN architectures, the Hessian matrix computed using pre-softmax is zero or None. This is due to the fact that operations like ReLU and convolution are linear. The Hessian only becomes non-zero when a nonlinear activation function is involved (such as EfficientNet) or using other utility function (such as post-softmax or the mentioned ReST).

For more details, please check this response:
#550 (comment)

[hxngu]

Thanks a lot for your quick reply.