Reproduce the results of the baseline in SIM 10k to cityscape

[xiong233]

I am planning to reproduce the results of the baseline in SIM 10k to cityscape on Pytorch, but I got a very high result of about 42%. And dafaster got a normal result of about 40%.

In fact, I have similar results from KITTI to cityscape. I may not be right, but I can't find the reason. Is there any detail to be aware of when implementing the baseline? Thank you!

[xiong233]

I trained on the baseline with the SIM 10k dataset and tested the car AP on the cityscape to get 42% of the results.

I tested it on the foggy cityscape, but got the normal result of 26%.

Should I use foggy cityscape as a test set?

[krumo]

Hi @xiong233 , are you using the pretrained vgg16 weights from torchvision? As far as I know, VGG16 pretrained weight in torchvision may yield weird performance. Here is an example. So my suggestion would be

1. try VGG16 weight converted from caffe.
2. switch to resnet50 to see whether this phenomena still exists.

Plus, if you are reproducing sim10k->cityscapes task, you should NOT use foggy cityscapes dataset. Looking forward to your results.

[xiong233]

嗨@ xiong233，您使用的是来自torchvision的预训练vgg16重量吗？据我所知，VGG16在火炬之前的预制重量可能会产生奇怪的性能。这是一个例子。所以我的建议是

1. 尝试从caffe转换的VGG16重量。
2. 切换到resnet50，看看这种现象是否仍然存在。

另外，如果您正在复制sim10k-> cityscapes任务，则不应使用模糊城市景观数据集。期待您的结果。

Thank you for your answer! I used the vgg pre-training model for caffe conversion, but got a 42% over-high result on the cityscape (not the foggy cityscape) on the baseline, which exceeded the result of da-faster-rcnn. I will try if resnet50 will still be so high