IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing

[Project] [Paper]

IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing,
Chun Gu, Xiaofei Wei, Zixuan Zeng, Yuxuan Yao, Li Zhang
CVPR 2025

Official implementation of "IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing".

demo_video_github_20241220_18142463.mp4

🛠️ Pipeline

⚙️ Installation

git clone https://github.com/fudan-zvg/IRGS.git

# This step is same as 2DGS/3DGS
# Please be aware that the submodules/diff-surfel-rasterization is slightly different from the original version in 2DGS.
conda env create --file environment.yml
conda activate irgs

# Install diff-surfel-rasterization and simple-knn
pip install submodules/diff-surfel-rasterization submodules/simple-knn

# Install raytracing (for Ref-Gaussian in stage 1)
pip install submodules/raytracing

# Install 2D Gaussian Ray Tracer
cd submodules/surfel_tracer && rm -rf ./build && mkdir build && cd build && cmake .. && make && cd ../ && cd ../../
pip install submodules/surfel_tracer

📦 Dataset

Download the Syn4Relight dataset from LINK provided by InvRender.

Download the TensoIR dataset from LINK and Environment maps from LINK provided by TensoIR.

Put them under the data folder:

data
└── Synthetic4Relight
    └── air_baloons
    └── chair
    └── hotdog
    └── jugs
└── TensoIR_Synthetic
    └── armadillo
    └── Environment_Maps
    └── ficus
    └── hotdog
    └── lego

Training

See run_syn4relight.sh and run_tensoir.sh for training and evaluation scripts on Synthetic4Relight and TensoIR datasets, respectively.

Stage 1: geometry reconstruction

We recommand to use our group's Ref-Gaussian for robust geometry reconstruction.

CUDA_VISIBLE_DEVICES=0 python train_refgaussian.py -s data/Synthetic4Relight/jugs -m outputs/Synthetic4Relight/jugs/refgs --eval -w --lambda_mask_entropy 0.05

Stage 2: material decomposition

CUDA_VISIBLE_DEVICES=0 python train.py -s data/Synthetic4Relight/jugs  --iterations 20000 --start_checkpoint_refgs outputs/Synthetic4Relight/jugs/refgs/chkpnt50000.pth --envmap_resolution 128 --lambda_base_color_smooth 2 --lambda_roughness_smooth 2 --diffuse_sample_num 256 --envmap_cubemap_lr 0.01 --lambda_light_smooth 0.0005 --init_roughness_value 0.6 --lambda_light 0.1 -m outputs/Synthetic4Relight/jugs/irgs --train_ray

Evaluation

Evaluation on Synthetic4Relight dataset

# Evaluate NVS, render various kinds of images
CUDA_VISIBLE_DEVICES=0 python render.py -m outputs/Synthetic4Relight/jugs/irgs --eval --diffuse_sample_num 512

# Compute albedo scale for alignment
CUDA_VISIBLE_DEVICES=0 python compute_albedo_scale_syn4.py -m outputs/Synthetic4Relight/jugs/irgs

# Evaluate the decomposed material (albedo, roughness)
CUDA_VISIBLE_DEVICES=0 python eval_material_syn4.py -m  outputs/Synthetic4Relight/jugs/irgs --albedo_rescale 2

# Evaluate the relighting performance
CUDA_VISIBLE_DEVICES=0 python eval_relighting_syn4.py -m outputs/Synthetic4Relight/jugs/irgs --diffuse_sample_num 512 --light_sample_num 256 --albedo_rescale 2 -e light

Evaluation on TensoIR dataset

# Evaluate NVS, render various kinds of images
CUDA_VISIBLE_DEVICES=0 python render.py -m outputs/TensoIR_Synthetic/armadillo/irgs --eval --diffuse_sample_num 512

# Compute albedo scale for alignment
CUDA_VISIBLE_DEVICES=0 python compute_albedo_scale_tensoir.py -m outputs/TensoIR_Synthetic/armadillo/irgs

# Evaluate the decomposed material (albedo, normal)
CUDA_VISIBLE_DEVICES=0 python eval_material_tensoir.py -m outputs/TensoIR_Synthetic/armadillo/irgs --albedo_rescale 2

# Evaluate the relighting performance
CUDA_VISIBLE_DEVICES=0 python eval_relighting_tensoir.py -m outputs/TensoIR_Synthetic/armadillo/irgs --diffuse_sample_num 512 --light_sample_num 256 --albedo_rescale 2 -e light   

📜 BibTeX

@article{gu2024IRGS,
  title={IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing},
  author={Gu, Chun and Wei, Xiaofei and Zeng, Zixuan and Yao, Yuxuan and Zhang, Li},
  booktitle={CVPR},
  year={2025},
}