Frequent ID swapping (How to improve proofreading efficiency in this case?)

[sdfclaj123]

We have already trained a good animal model and it worked well when we ran inferences on two test videos (one 2-minute video and one 10-minute video; 30fps; 12 flies in each video): accurately located each fly and very minimal ID swapping. However, when we ran inferences on the experiment videos (exact same setup as the test videos), we started to have a lot of ID swapping when one fly is chasing another. These flies are not really interacting (no body parts overlapped) and ID swapping does not only occurs when they are on the side but also when they are in perfectly flat poses. What would be the reason for this? What can we do to improve? As we have hundreds of videos to process, this ID swapping issue is very problematic.

[talmo]

Hi @sdfclaj123,

Apologies for the delay!!

Since the identity tracking isn't learnable, it shouldn't be impacted by a domain gap/failure to generalize on new videos, so if the poses are still looking good on the new data, the identity tracking should work just as well.

Do you mind sharing the tracking parameters or CLI command that you're using?

Could you also share a clip of when an ID switch happens on the new data? (You can hold shift while dragging across the seekbar to select a clip, then render a video with the poses from the View menu.)

Cheers,

Talmo

[sdfclaj123]

Sorry for the late response!

This would be the setting for running inference. We never get it working with "flow" so we use "simple".

Here is one example clip of the ID swapping (top left arena, the never-ending swapping of the two instances green and purple)

Is there something we can try?

[talmo]

Hi @sdfclaj123,

Thanks for sharing the clip!

Indeed that looks problematic and unexpected.

A few questions:

1. What happens when you turn the tracking window down to 3?
2. Have you tried using the max instead of the robust quantile?
3. Disabling connect single track break?
4. Using instance similarity instead of IOU?
5. Would you mind sharing the SLP file and video with us so we can do some troubleshooting on our end?

Cheers,

Talmo