Add Evo2 Genomic foundation model

[McClain-Thiel]

Model description

Evo2 is a Decoder only StripedHyena2-based causal LM that models DNA in the same fashion frontier models are based on pretrained only models. Huggingface has similar models available but Evo2 is state of the are on long range dependency modeling and down stream variant prediction.

Evo2 in huggingface would allow Bio x ML folks to experiment with many down stream tasks such as variant prediction, cancer modeling, genomic design and more.

Evo2 uses a DNA base pair tokenizer.

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

@ArcInstite originally implemented a non HF compatible version here: https://github.com/ArcInstitute/evo2

[Rocketknight1]

Hi @McClain-Thiel, definitely interested! Linking the PR at #42210 as well

[shanto12]

• Adding Evo2 to transformers would be valuable for the computational biology and bioinformatics community. Genomic foundation models are a growing field and having HF-native support would lower barriers for researchers.

Integration Considerations:

• The DNA base pair tokenizer is unique compared to standard BPE/WordPiece tokenizers - ensure proper tokenizer abstraction
• Long-range dependency modeling (Evo2 handles long genomic sequences) may require special attention to memory optimization
• Document genomic-specific use cases (variant prediction, cancer modeling, genomic design) to help bio researchers discover the model

Community Impact:
This would bridge the ML and bio communities, similar to how ESM models for protein sequences gained traction through HF. The implementation would set a template for future genomic models.