Interested in contributions?

[andreashhpetersen]

Hi! I am really interested in this project as the VIPER algorithm is relevant for my own research (which is also within safe and explainable RL). Therefore I would like to know, if you are interested in contributions?

I've encountered a couple of errors/warnings when running the code as-is and I was thinking of opening issues and pull requests for these (train/viper.py because of torch device related issues in line 125/126, and test/evaluate.py throws a lot of UserWarnings since the environments don't get wrapped in a Monitor class).

For my own project, I want to be able to supply a custom oracle policy (ie. one that is not necessarily trained as part of the pipeline of this package and maybe is based on a different model than DQN or PPO). Would you be interested in contributions of that sort?

[chrismatix]

Hi @andreashhpetersen,

yes, I would be very interested in this and contributions are always welcome. However, I am doing this implementation as part of a university course. So to not interfere with grading could you wait with pull requests to this repository until the beginning of March? When it comes to new algorithms I guess the only constraint that we currently have is that they work with SB APIs.

I've encountered a couple of errors/warnings when running the code as-is and I was thinking of opening issues and pull requests for these (train/viper.py because of torch device related issues in line 125/126, and test/evaluate.py throws a lot of UserWarnings since the environments don't get wrapped in a Monitor class).

Apologies for this. I am still mostly developing locally and will push a new version tomorrow.
Again thank you so much for also being excited about this paper and wanting to build a good implementation!

[andreashhpetersen]

Sounds great @chrismatix ! I'll wait with any PRs untill you are ready, and don't sweat it fixing anything for my sake (I have a working copy now and I am just toying around to get my own results).

But let me know when you are up for contributions, as it would be cool to have a working implementation of this algorithm running.

[chrismatix]

Hi @andreashhpetersen,
so my course just concluded and if you are still interested we could collaborate on this project. I did a brief writeup of the main experiment here: https://safe-rl-team.github.io/viper-verifiable-reinforcement-learning/

There are some small details I contacted the main author about; otherwise, this implementation is working. However, the main part of the algorithm is very brief so I am not sure it is worth turning it into a package or something to add to one's codebase. What do you think?