Details regarding optical flow pre-processing

[simonguiroy]

Hi Yana! Do you have some details on the your pre-processing for the optical flow? I've tried using your model with my own pre-processing that worked with the original TensorFlow repo of I3D, but when using your implementation of I3D the predictions are totally off when I use the flow stream.

[hassony2]

Hi Simon,

As you might see here, I don't do anything specific beyond loading their sample in my code.

I therefore don't know what might cause this discrepancy.

Let me know if you find out where that came from !

Yana

[simonguiroy]

Hi Yana, I encountered another problem when using your code. I after creating the model, if model.train() is called, training accuracy drops significantly, compared to when model.eval() is called, even without `with torch.no_grads()`. Could there be something with the batchnorm layer implementation, that is linked to that behavior? Thanks again Simon

…

On Thu, 6 Sep 2018 at 12:12, Yana ***@***.***> wrote: Hi Simon, As you might see here <https://github.com/hassony2/kinetics_i3d_pytorch/blob/a4b815fb95d311bbf01734f4d1d3610f7f39cde1/i3d_pt_demo.py#L35>, I don't do anything specific beyond loading their sample in my code. I therefore don't know what might cause this discrepancy. Let me know if you find out where that came from ! Yana — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APn0cDhMjGMbXVAgM5KuKSlxB4DwlrMJks5uYUmHgaJpZM4WdODu> .

[gurkirt]

It is probably due to batch normalisation layers acting up. If you want to evaluate new videos then use .eval(), If you want to train the model with a smaller batch size then freeze the normalisation layer.

[ed-fish]

See this issue which details optical flow pre-processing on the original Tensorflow implementation. google-deepmind/kinetics-i3d#87