Unable to reproduce results using refining checkpoints available in the repo

[sourabsridhar]

Hello,

I am trying to reproduce the results from the paper on the validation subset of the Waymo dataset using the refining checkpoints available in the Google Drive link, and the best numbers that I can reach after Geometric and Positional refining are as follows:

Note: These numbers were generated by running the detzero_eval.py code as per the README instructions.

The numbers here seem to be way lower than the numbers reported in the paper:

For the detector, I use the detections on the same validation subset of the Waymo dataset which already available from the FSDv2 paper [Link to the paper] [Link to the Waymo detections], where the FSDv2 detector has a comparable detection results reported for vehicles when compared to detection results reported for vehicles by the DetZero paper. These comparison between these results are attached below:

FSDv2 detection results on Waymo dataset as reported in the paper:

DetZero detection results on Waymo dataset as reported in the paper:

For the tracker and the refining, I use the settings from the paper/repo as it is, and I do not have any additional changes.

Do you have any suggestions on what could be the problem here?

Thanks a lot!

[CodeSageMaster]

Hello, can you tell me how to use the checkopint published by the author to make predictions on the test set, thanks

[Annie990401]

Hi，have you successfully inferred the correct results using the pedestrian and cyclist GRM checkpoints? I tried using these checkpoints for inference, but the bbox sizes are not correct.

[llz666ha]

Hello, is the dataset you are using the complete Waymo dataset? What is the version?