Feature Request: Improve SAM2 Tracking in Dense Object Scenarios to Avoid Full Resets

[SongJgit]

Search before asking

• I have searched the X-AnyLabeling Docs and issues and found no similar questions.

Question

hello，你好，我有些问题想要咨询一下，关于sam2进行交互式多目标跟踪标注的。
我发现了一个问题，如果是密集场景的话，有新目标加入，就要清空当前帧的所有目标然后重置跟踪器。但是这样有些太麻烦了，密集目标可能有几十上百个，每一个都重新标注还需要和之前的帧对齐id。有没有什么解决方案可以尽可能的跳过这个重新标注已有目标的方法。
尝试了复制前一帧标签的方法，但是似乎这样，SAM2不会跟踪已有目标

Additional

No response

[CVHub520]

Hello! @SongJgit, Thank you for bringing this to our attention. You're absolutely correct; currently, when a new target appears in a dense scene, it requires clearing all existing targets in the current frame and resetting the tracker. This is indeed not ideal for dense object tracking scenarios.

The core issue lies in correctly passing the current coordinate information to the self.prompts variable.

Unfortunately, I likely won't have the bandwidth to optimize this in the near future. If you're able to, we'd greatly appreciate it if you could attempt to implement a solution and submit a Pull Request (PR)! I'm happy to provide assistance and guidance.

Specifically, you can focus on this section of the code.

If you have any further questions, please don't hesitate to ask! 😄

[FinalYou]

Hi, I was going through this thread and also looked into that section of the code.

I encountered a similar situation while doing tracking annotations. When propagating to a certain frame, some annotations remain correct while others are wrong. I need to fix the wrong ones and then click “reset tracker” to continue tracking, but this clears all annotations on that frame and forces me to re-annotate everything, even the correct ones.

So, my case is similar. If there were a function like “reset tracker based on current frame” that directly takes the current frame masks and converts them into new marks/prompts for the tracker, that would be awesome.

Any tips or workarounds would be appreciated!