Evalution for Qwen2

[kris-singh]

Hi,
Thanks for the great work!

I have a couple of questions.

1. The README.md file says you used the conv_mode=phi for training, while the train_qwen2_base.sh file has the conv_mode=qwen2_base. Is this a typo in the README file?
2. To evaluate models trained with the Qwen2 model, what should the conv_mode argument be set to? Is it qwen2_base or qwen2_instruct? I am assuming that it is qwen2_instruct.
3. Finally, what should one consider for evaluating the model on more benchmarks, such as SugarCrepe? Is it just the conv_mode parameter or something else as well?

Update:

1. This is a typo in the README file.
2. It turns out you need to use qwen2_base for evaluation.

Moreover, I found out a bigger bug with the script files. The Phi-2 model has a max length of size 2048 and not 3072.

Thanks,
Krish

[ZhangXJ199]

1. conv_mode is related to the type of language model. When using Qwen as the language model, use qwen2_base.
2. It depends on whether you're using the base model or the instruct model of Qwen2.
3. As mentioned above, conv_mode is only related to the language model.
4. We have changed it to 2048.