Propagate and track multiple objects

[ivanlyubimtsev]

Is it possible with your current codebase to propagate and track multiple objects simultaneously in a video (with different object IDs), not just a single one? Or is the framework currently limited to single-object propagation when using the VLM module?

[felipeanndrade]

I'm trying to achieve the same purpose. The approach that I'm trying is to use multiple masks and iterate for every single one. Since my purpose is to analyse behaviour, I'm making a cluster using the output and cross-comparing them.

[ivanlyubimtsev]

I'm trying to achieve the same purpose. The approach that I'm trying is to use multiple masks and iterate for every single one. Since my purpose is to analyse behaviour, I'm making a cluster using the output and cross-comparing them.

Just wondering, what limits SeC from propagating multiple objects at once?
is it mainly a memory thing, or something in how the VLM branch works?

[felipeanndrade]

I'm not 100% sure. I would guess that it's the way that it was implemented for the research.

What I mean is: Given an image, LVLMs can describe multiple objects at once. Therefore, applying the concept guidance to multiple objects is achievable (not easy, nor computationally efficient/cheap).