Support for MiniWorld (3D indoor environment)?

[maximecb]

Hi Lucas,

I've been working on my 3D indoor environment. It's still very basic, but it works, and I just made the repository public: https://github.com/maximecb/gym-miniworld

I've tried to adjust your pytorch-a2c-ppo code to work with MiniWorld, but ran into issues. One is that MiniWorld produces observations which are not dictionary-based, and it was awkward to support this (the obs are just 60x80x3 RGB arrays). The other is that I only got something like 16 frames per second while training with 16 processes, and I have no idea why.

Would you have time to take a look? It would be great if it could work with your RL code out of the box. Right now I'm using my own fork of ikostrikov's, but there are multiple other issues with that code, one of which is that the performance when visualizing trained agents doesn't match the performance reported while training.

[lcswillems]

Hi Maxime,

This is a really great project!!!

For the first issue, is it still an issue after I modified the network to automatically adapt it to the size of the images?

Thank you for the issues. I will try your code as soon as possible.

[maximecb]

Thanks Lucas. I will keep improving upon it :)

I tried the following comman for training:

python3 -m scripts.train --algo ppo --env MiniWorld-Hallway-v0 --procs 1 --no-instr --no-mem --save-interval 10

It produces the following error:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/maximecb/Desktop/pytorch-a2c-ppo/scripts/train.py", line 111, in <module>
    preprocess_obss = utils.ObssPreprocessor(model_dir, envs[0].observation_space)
  File "/home/maximecb/Desktop/pytorch-a2c-ppo/utils/format.py", line 40, in __init__
    "image": obs_space.spaces['image'].shape,
AttributeError: 'Box' object has no attribute 'spaces'

Because it expects observations to have a dictionary observation space.

I think it would be good to support both dict and RGB observation spaces. This would allow your RL code to work not just with MiniWorld, but also with the Atari environments.

[lcswillems]

If I assume that the obs_space is a dictionnary, it is because the MiniGrid's observation space is Dict(image:Box(7, 7, 3)). The easiest thing would be to change MiniGrid's observation space to be just Box(7, 7, 3). This way, the observation space of MiniGrid, MiniWorld and Atari would be Box(...) and hence, I will not have to do disjunctions.

[maximecb]

Yes, if I had to do it again I would make the MiniGrid obs space just a numpy array, and pass the mission string through the info dict instead. I'm kind of reluctant to do that now though because some number of people are already using MiniGrid, and this change could break their code.

[lcswillems]

I see what you mean. So, you will never do changes that are no retro compatible?

Are their instructions in MiniWorld?

[maximecb]

I will try to avoid breaking changes if I can.

Right now there are no instructions in MiniWorld, but there will be eventually. When there are instructions I will add them through the info dict instead, it seems like the better way to go with OpenAI Gym.

[lcswillems]

I would like to do a clean code that would define a observation space based on the kind of environment, i.e. if it is a minigrid environment, then ..., else if it is a MiniWorld environment then, otherwise ... .

Do you have an idea of how to do that?

For example, is it possible to get the id of the environment? I don't find how to get it... If yes, then I could filter using regexs.

[maximecb]

You could check if the observation space is an instance of Box or if it's an instance of Dict

https://github.com/openai/gym/blob/master/gym/spaces/dict_space.py

[lcswillems]

Thank you for the solution. But doing the disjunction on the observation space is not the good thing to do I think. It doesn't make the code easy to understand.

[maximecb]

The good thing is that if you check based on the observation space, you could handle not just MiniWorld, but also all other environments which produce an RGB pixel array as observations (eg: Atari, VizDoom, etc).

[lcswillems]

What is the FPS you get with the RL code you actually use?

[maximecb]

1600-2200FPS on a Core i7 3.5 with a Titan Xp, 16 processes.

[lcswillems]

Sorry, I meant, what is the FPS you get with the other RL code you use (not mine)? https://github.com/maximecb/gym-miniworld/tree/master/pytorch-a2c-ppo-acktr

[maximecb]

Yes that's what I meant.