ValueError: Image features and image tokens do not match: tokens: 1991, features 2990

[zhangboshen]

Related discussion : huggingface/transformers#34625
I met this problem when sft qwen2-vl-7b using lora, training data is multi-images QA.

[CloudComputing-linux]

same problem

[thusinh1969]

What happened ? I used to be able to finetune Qwen2-VL with multiple images just fine. What is the bug now ?!?!

Cheers,
Steve

[gouqi666]

@zhangboshen @thusinh1969 @CloudComputing-linux @hzhwcmhf @c0g anyone have solution for this mismatch problem? i use llamafactory to train qwen2-vl

[mrwwk]

same

[zhangboshen]

@thusinh1969 @gouqi666 @CloudComputing-linux @mrwwk
I'm able to train the model by setting cutoff_len to a larger num that default. e.g., 10000;
It seems like this parameter affect the image token-image feature alignment to some extent.

[koleshjr]

Same I am facing this issue. Any fixes?