Grounding

[SFTJBD]

Thanks for the new model. I've made several attempts with the grounding task, but I still haven't seen <|box_start|> and <|box_end|> IDs in the output_id. I'm wondering if the current version of the grounding task has discontinued the use of <|box_start|> and <|box_end|> as special tokens, and the training method of writing coordinates in between them?

[HumanZhong]

@SFTJBD
Hello, Qwen2.5-VL does not use special tokens for grounding during SFT, as a result you will not see any grounding-related special tokens when trying Qwen2.5-VL's grounding capability. Instead, we use JSON/XML/Plain-Text format to output bbox/point coordinates for easy results parsing.

[maryhh]

@SFTJBD Hello, Qwen2.5-VL does not use special tokens for grounding during SFT, as a result you will not see any grounding-related special tokens when trying Qwen2.5-VL's grounding capability. Instead, we use JSON/XML/Plain-Text format to output bbox/point coordinates for easy results parsing.

So, during fine-tuning training, I don't need to include <|box_start|> and <|box_end|> A special tokens anymore, right？
like this？
{"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "描述图像"}, {"role": "assistant", "content": "<|object_ref_start|>一只狗<|object_ref_end|>(221,423),(569,886)和<|object_ref_start|>一个女人<|object_ref_end|>(451,381),(733,793)}], "images": ["/xxx/x.jpg"]}
can you provide some data examples for grounding task fine-tuning?thank you.

[SFTJBD]

@SFTJBD Hello, Qwen2.5-VL does not use special tokens for grounding during SFT, as a result you will not see any grounding-related special tokens when trying Qwen2.5-VL's grounding capability. Instead, we use JSON/XML/Plain-Text format to output bbox/point coordinates for easy results parsing.

Thank you for your answer! I have another question. In the task of temporal grounding for videos (for example, providing timestamps), are the annotations also in terms of absolute timestamps? Are there any strict requirements or corresponding relationships between the FPS or nframes during training and inference?