Which evo2 model to select for finetuning #1315
Description
Description
We are currently finetuning evo2 on token level classification. Thanks to your help here #1170 we've managed to move along and finetune the evo2/1b-8k:1.0 model.
However, we noticed there are two checkpoints available for the Evo2 base model — evo2/1b-8k:1.0 and evo2/1b-8k-bf16:1.0. From the descriptions, it seems that the first is the original NeMo2-format conversion of arcinstitute/savanna_evo2_1b_base, while the second is a fine-tuned variant optimized for BF16 precision. For the fairest comparison to Evo2, could you advise which checkpoint and training precision you recommend using?