Windows Instruction for Blackwell GPU

[Katehuuh]

Win installation-python 3.10, RTX 50xx only supports CUDA≥12.8:

git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git && cd LLaMA-Factory
python -m venv venv
call venv\Scripts\activate.bat
python -m pip install --upgrade pip
pip install -e ".[torch,metrics]"
pip install torch==2.7.0+cu128 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 --no-cache --force-reinstall --version --isolated --ignore-installed --no-cache-dir --upgrade

pip install -U bitsandbytes
pip install -U triton-windows

:: Any flash_attn WHL in https://github.com/kingbri1/flash-attention/releases
pip install https://github.com/kingbri1/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu128torch2.7.0cxx11abiFALSE-cp310-cp310-win_amd64.whl

llamafactory-cli webui

Quick Start example:

1. Model name: Llama-3.1-8B-Instruct - Will use 10GB VRAM
   1.1 Without HF tokens change Model path: meta-llama/Llama-3.1-8B-Instruct to NousResearch/Meta-Llama-3.1-8B-Instruct
2. Quantization bit: 4 - QLoRA
3. Tab Train > Dataset > alpaca_gpt4_en is selected, which is a Q&A dataset located in ./data.
4. we'r now good:[Start]
   After train LoRa/adapter is in ./saves, to merge back to model use Export Tab or else, directly Chat with the selected checkpoint path.