Low Success Rate When Using Diffusion Policy on the StackCube-v1 Task

[ZhangHangTao]

I’ve noticed that the RGB-based diffusion policy fails on many tasks, which is similar to what the authors reported: https://wandb.ai/stonet2000/ManiSkill/workspace?nw=3d49kizu62e
.

I’m sorry to raise this, but I think the very low success rate is due to an issue in the code rather than an insufficient dataset or other factors. From the training logs you provided, for the diffusion policy, the state-based results achieve very high success rates, whereas the RGB-based results are consistently low across many tasks, which is counterintuitive. For example, why is the success rate for PickCube significantly higher than for StackCube? I don’t think the two tasks should differ that much. I’ve combed through almost all the issues and PRs, and StackCube seems to be a nightmare for the diffusion policy.

In particular, after reviewing many inference videos, I noticed it often fails to localize the correct position, consistently deviating to the left or to the right.