v0.1.0

We are releasing ZO2 v0.1.0, the initial public version of our high-throughput, memory-efficient zeroth-order fine-tuning framework for large language models.

Although still in early development (pre-1.0), this release contains all core features to reproduce the main results of our ZO2 paper.

🚀 Highlights

• Full-parameter fine-tuning with only 18GB GPU for models up to 175B
• CPU offloading for gradients and optimizer states
• Hugging Face Transformers compatibility
• Minimal codebase for educational and experimental use

🧪 Tested Models

• OPT-125M to OPT-175B

📄 Getting Started

  git clone https://github.com/liangyuwang/zo2.git
  cd zo2/
  conda env create -f env.yml
  conda activate zo2