Reproduction Results Inconsistent with Paper

[YsTvT]

Dear Authors,
I attempted to reproduce the results from your paper, but encountered significant discrepancies. When running vagen_base/sokoban, I obtained a result of 0.156, which is much lower than the reported 0.38 in the paper. Similarly, for masked_turn_ppo/frozenlake, my reproduction yielded 0.344 compared to the paper's 0.71. Could you please help clarify why there might be such large differences? Are there any specific hyperparameter settings, configuration files, or additional training details that need to be modified or considered that might not be fully documented in the current repository? I would greatly appreciate any guidance on reproducing the reported results.
Thank you for your time and for open-sourcing this work.
Best regards

[JamesKrW]

Thank you for reaching out and for taking the time to reproduce our results.
Could you please provide more details about your reproduction, such as logs/scripts/platform/? I will re-test the scripts on our side, but I believe we already performed several reproductions before. You can check our experiment logs and the figures attached below (ignore the naming differences, they all fall under the vagen-base setting). There may be some variance across runs, but it should not be that large.