colab tutorial？

[yejy53]

Thank you for your analysis, can you post a colab tutorial?

[rayleizhu]

It is a good suggestion. However, it takes quite some effort to tidy up the visualization code and build a nice demo. I'm not sure when I will have the motivation to finish it, since I'm chasing the next CVPR deadline, and there are too many things to do.

I can give some tips for the implementation if you would like to do it by yourself:

1. You can use register_forward_hook() to easily get the routing region indices and attention matrix, without modifying the model itself.
2. You can use cv2.applyColorMap to convert attention score map to pseudo-colored heat map.
3. You can then alpha blend the image and heatmap (i.e. alpha*image + (1-alpha)*heatmap) to get the visualization figure.

[yuxiongl]

Hello author, I am a newcomer in this field. Could you tell me the specific code for using [register_forward_hook()]. The model stages are nested too deeply. It is difficult for me to figure out the layer name. I don't know how to keep pointing out the middle value like model. bla. bla. bla.