Ground truth point clouds for intermediate and advanced test sets

[320160040]

Thank you for providing the evaluation code along with the ground truth data for training sets. In the python evaluation code, you provide links for downloading point clouds created with COLMAP and ground truth ply files. For example, for Ignatius set, you provide Ignatius.ply & Ignatius_COLMAP.ply. Here the Ignatius.ply is the ground truth point cloud while Ignatius_COLMAP.ply is created with COLMAP's 3D reconstruction. The evaluation code for calculating F-Score works fine with provided training sets.

However, I cannot find the ground truth data for intermediate and advanced test sets anywhere. Can you please suggest how/where to look for it?

[arknapit]

This is on purpose, we try to keep the ground truth hidden from the public for the intermediate and advanced test sets. This ensures, that people are not using this data to cheat our benchmarking system.

[zhao-you-fei]

@arknapit
So how should I evaluate the results of running the advanced dataset myself.
Thank you

[arknapit]

@zhao-you-fei If you want to run on the advanced-set, which is part of our test set, submitting to the evaluation server is the only way (see instructions on our site: https://www.tanksandtemples.org/submit/). If you want to run it yourself, working with the training data is the only way. for this, see instructions here: https://github.com/isl-org/TanksAndTemples/tree/master/python_toolbox/evaluation

[zhao-you-fei]

@arknapit
感谢您解答，我想问python run.py --dataset-dir path/to/TanksAndTemples/evaluation/data/Ignatius --traj-path path/to/TanksAndTemples/evaluation/data/Ignatius/Ignatius_COLMAP_SfM.log --ply-path path/to/TanksAndTemples/evaluation/data/Ignatius/Ignatius_COLMAP.ply中“Ignatius.py”是作为真值？“--traj-path和--ply-path”是什么意思，评估对象一定要是用colmap生成的点云吗？因为我没有看到自己用colmap或者mvsnet生成的点云数据中心有xx_COLMAP.ply和xx_COLMAP_SfM.log