Junliang Ye1,2*,
Fangfu Liu1*,
Qixiu Li1,
Zhengyi Wang1,2,
Yikai Wang1,
Xinzhou Wang1,2,
Yueqi Duan1†,
Jun Zhu1,2†,
*Equal Contribution.
†Corresponding authors.
1Tsinghua University,
2ShengShu,
- [2025/3/19] 🔥🔥We released the DreamFL code of DreamReward.
- [2025/3/19] 🔥🔥We open-source two versions of Reward3D weights.
We open-source two versions of Reward3D: Reward3D_Scorer, corresponding to the implementation presented in our paper, and Reward3D_CrossViewFusion, an extended version leveraging self-attention to enhance multi-view feature integration. Furthermore, we release two hyperparameter configurations of the DreamFL algorithm: DreamReward_guidance1, offering more conservative guidance, and DreamReward_guidance2, providing more aggressive guidance. Additionally, we provide ablation for further analysis and reference.
Our environment has been tested on Ubuntu 22, CUDA 11.8 with A100, A800.
- Clone our repo and create conda environment
git clone https://github.com/liuff19/DreamReward.git && cd DreamReward
conda create -n dreamreward python=3.9
conda activate dreamreward
pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu118
pip install xformers==0.0.23.post1
pip install ninja
git clone https://github.com/bytedance/MVDream extern/MVDream
pip install -e extern/MVDream
pip install -r requirements.txt
pip install fairscale
- Install the pretrained Reward3D weight
pip install -U "huggingface_hub[cli]"
huggingface-cli login
huggingface-cli download yejunliang23/Reward3D --local-dir checkpoints/
# inference for CrossViewFusion1
python launch.py --config "configs/dreamreward1.yaml" --train --gpu 0 system.guidance.alg_type="Reward3D_CrossViewFusion" system.guidance_type="DreamReward-guidance1" name="CrossViewFusion1" system.prompt_processor.prompt="A delicate porcelain teacup, painted with intricate flowers, rests on a saucer" system.guidance.reward_model_path="checkpoints/Reward3D_CrossViewFusion.pt"
# or you can
bash scripts/cvf1.sh
# inference for CrossViewFusion2
python launch.py --config "configs/dreamreward2.yaml" --train --gpu 0 system.guidance.alg_type="Reward3D_CrossViewFusion" system.guidance_type="DreamReward-guidance1" name="CrossViewFusion2" system.prompt_processor.prompt="A delicate porcelain teacup, painted with intricate flowers, rests on a saucer" system.guidance.reward_model_path="checkpoints/Reward3D_CrossViewFusion.pt"
# or you can
bash scripts/cvf2.sh
# inference for Scorer1
python launch.py --config "configs/dreamreward1.yaml" --train --gpu 0 system.guidance.alg_type="Reward3D_Scorer" system.guidance_type="DreamReward-guidance1" name="Scorer1" system.prompt_processor.prompt="A delicate porcelain teacup, painted with intricate flowers, rests on a saucer" system.guidance.reward_model_path="checkpoints/Reward3D_Scorer.pt"
# or you can
bash scripts/scorer1.sh
# inference for Scorer2
python launch.py --config "configs/dreamreward2.yaml" --train --gpu 0 system.guidance.alg_type="Reward3D_Scorer" system.guidance_type="DreamReward-guidance1" name="Scorer2" system.prompt_processor.prompt="A delicate porcelain teacup, painted with intricate flowers, rests on a saucer" system.guidance.reward_model_path="checkpoints/Reward3D_Scorer.pt"
# or you can
bash scripts/scorer2.sh
- Release the 3DReward-training code.
Our code is based on these wonderful repos:
Also we invite you to explore our latest work DeepMesh about applying direct preference optimization to 3D-mesh generation.
@inproceedings{ye2024dreamreward,
title={Dreamreward: Text-to-3d generation with human preference},
author={Ye, Junliang and Liu, Fangfu and Li, Qixiu and Wang, Zhengyi and Wang, Yikai and Wang, Xinzhou and Duan, Yueqi and Zhu, Jun},
booktitle={European Conference on Computer Vision},
pages={259--276},
year={2024},
organization={Springer}
}