Inference labels desynchronized from video frames?

[AlbertChenLab]

After training a model and generating reasonably accurate labels on random frames, I ran inference on the entire video. But it looks like the prediction labels precede the video by around 40 frames. Any suggestions as to what could be causing this? I never have this issue when running inference on random single frames, or predicting on 20 random frames. The only error i had in the run was

Failed to read frame index 14153

But that is the very last frame of the video and the only frame that SLEAP seemed to have failed to read according to the log. Could it be that there are other frames not being read and are skipped causing the labels to be transposed onto the wrong frames?

I've searched the forum and found one other instance of someone experiencing this but that post has no suggestions.

Video example:
https://github.com/user-attachments/assets/5a3e31cd-b041-439f-a68d-35c1c373f162

[gitttt-1234]

Hi @AlbertChenLab !

Could you check the fps of the video? I'm attaching a similar issue: #2149. We've seen this lag if the fps was not an integer.

Thanks,

Divya

[AlbertChenLab]

The frame rate was set at 200 fps during recording, but looking at the video properties it says "200.00". Would you expect the fps value being a float be the cause of the issue based on previous issues? Thank you

[AlbertChenLab]

Update- it looks like it was some sort of problem with the encoding of the video not allowing SLEAP to read frame indeces correctly. Here is an example of scrubbing through the video in GUI and "frame 1" displaying different frames:
https://github.com/user-attachments/assets/a23eee26-b067-4708-9bc0-820deebdb98a

I was able to resolve the issue by re-encoding the .avi file and saving as .mp4 using ffmpeg. The new .mp4 file did not have desync issue or frame display issues in the GUI. I used following settings:
ffmpeg -y -i input.mp4 -r 200 -vsync cfr -c:v libx264 -pix_fmt yuv420p -preset superfast -crf 23 output.mp4

Thank you for your help in pointing me to that similar issue!