Skip to content

Commit 4f05f0c

Browse files
MadcowDMadcowD
authored
MineRL 0.2.0 [rebased] (#136)
* Add ability to observe placement of non-agent placeable blocks * Fix loud broken pipe error on data loading framework * Add action_space and observation_space to data pipeline * Made obsevation space a member of data class * Add spaces to package * Add other type to handlers * test handlers * Add debug flags to data pipeline to reduce typical printing * Enable debug flag in pipeline * use logger log levels * Expose logger * Unexpose logger * Add queue size parameter * Add debug statements * debug for days * debug for days * Add ability to observe placement of non-agent placeable blocks * Fix loud broken pipe error on data loading framework * Add action_space and observation_space to data pipeline * Made obsevation space a member of data class * Add other type to handlers * test handlers * Add debug flags to data pipeline to reduce typical printing * use logger log levels * Expose logger * Unexpose logger * Add queue size parameter * Reveering changes. * Addding data versioning. * Updating Malmo for gamma fix. * Update load_data to use current environment * Update load_data function * Align data based on actual number of frames in recording * A workable stream viewer! * Making stream viewer a module in minerl. * Adding a new streamviewier script. * viewier. * Windows requirement.txt * Updating. * Updating one more time. * Importing on windows. * need more color * Updating your cool applicaton.xtxt * yeah dude! * remove debug * Added version support to download script * Updating the head to support the new viewer. * Updating viewer to support reward plots. * Updating the new data viewer. * Add ability to return metadata in seq_iter * Fix metadata * Fix metadata * Pushing. code. * Update load data * Add metadat if not there * Fix * Deprecating seq_iter. * Increase timeout in download script closes #131 * Remove teleport observation for final state * Allowing for random streams. * Fix minerl data download version * Fix minerl data download version * Update documentation for data pipeline * Update setup.py * Delete mc_1.log * Update documentation for data itter * Updating the documentation. * Updating Malmo version * Updating the viewer for the new version of python3.5. Why oh why matplotlib :( * Adding a dependency for pySmartDL. * Adding new updates. * Yes.
1 parent 913dd02 commit 4f05f0c

File tree

18 files changed

+922
-96
lines changed

18 files changed

+922
-96
lines changed

‎.gitmodules‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
11
[submodule "minerl/env/Malmo"]
22
path = minerl/env/Malmo
33
url = https://github.com/cmu-rl/Malmo.git
4+
[submodule "minerl/dependencies/pySmartDL"]
5+
path = minerl/dependencies/pySmartDL
6+
url = https://github.com/minerllabs/pySmartDL.git

‎docs/source/assets/cropped_viewer.gif‎

2.42 MB
Loading

‎docs/source/assets/minerl_viewer.gif‎

2.5 MB
Loading

‎docs/source/index.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ components:
7676
:caption: Tutorials and Guides
7777
:maxdepth: 2
7878

79-
tutorials/getting_started
79+
tutorials/index
8080
tutorials/first_agent
8181
tutorials/data_sampling
8282

‎docs/source/tutorials/data_sampling.rst‎

Lines changed: 81 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -17,25 +17,94 @@ Now we can build the datast for :code:`MineRLObtainDiamond-v0`
1717

1818
.. code-block:: python
1919
20-
data = minerl.data.make('MineRLObtainDiamond-v0')
20+
data = minerl.data.make(
21+
'MineRLObtainDiamond-v0',
22+
data_dir='/your/local/path')
2123
2224
# Iterate through a single epoch gathering sequences of at most 32 steps
23-
for obs, rew, done, act in data.seq_iter(num_epochs=1, max_sequence_len=32):
24-
print("Number of diffrent actions:", len(act))
25-
for action in act:
26-
print(act)
27-
print("Number of diffrent observations:", len(obs), obs)
28-
for observation in obs:
29-
print(obs)
30-
print("Rewards:", rew)
31-
print("Dones:", done)
25+
for current_state, action, reward, next_state, done \
26+
in data.sarsd_iter(
27+
num_epochs=1, max_sequence_len=32):
3228
29+
# Print the POV @ the first step of the sequence
30+
print(current_state['pov'][0])
3331
32+
# Print the final reward pf the sequence!
33+
print(reward[-1])
3434
35+
# Check if final (next_state) is terminal.
36+
print(done[-1])
3537
38+
# ... do something with the data.
39+
print("At the end of trajectories the length"
40+
"can be < max_sequence_len", len(reward))
3641
37-
.. note::
42+
43+
.. warning::
3844
The :code:`minerl` package uses environment variables to locate the data directory.
3945
For portability, plese define :code:`MINERL_DATA_ROOT` as
40-
:code:`/your/local/path/data_texture_0_low_res` in your system environment variables.
46+
:code:`/your/local/path/` in your system environment variables.
47+
48+
49+
50+
=============================================================
51+
Visualizing The Data :code:`minerl.viewer`
52+
=============================================================
53+
54+
To help you get familiar with the MineRL dataset,
55+
the :code:`minerl` python package also provides a data trajectory viewer called
56+
:code:`minerl.viewer`:
57+
58+
59+
.. image:: ../assets/cropped_viewer.gif
60+
:width: 90 %
61+
:alt:
62+
:align: center
63+
64+
65+
The :code:`minerl.viewer` program lets you step through individual
66+
trajectories,
67+
showing the observation seen the player, the action
68+
they took (including camera, movement, and any action described by an MineRL
69+
environment's action space), and the reward they received.
70+
71+
.. exec::
72+
73+
import minerl
74+
import minerl.viewer
75+
76+
help_str = minerl.viewer.parser.format_help()
77+
78+
print(".. code-block:: bash\n")
79+
for line in help_str.split("\n"):
80+
print("\t{}".format(line))
4181

82+
83+
**Try it out on a random trajectory by running:**
84+
85+
.. code-block:: bash
86+
87+
# Make sure your MINERL_DATA_ROOT is set!
88+
export MINERL_DATA_ROOT='/your/local/path'
89+
90+
# Visualizes a random trajectory of MineRLObtainDiamondDense-v0
91+
python3 -m minerl.viewer MineRLObtainDiamondDense-v0
92+
93+
94+
95+
**Try it out on a specific trajectory by running:**
96+
97+
.. exec::
98+
99+
import minerl
100+
import minerl.viewer
101+
102+
traj_name = minerl.viewer._DOC_TRAJ_NAME
103+
104+
print(".. code-block:: bash\n")
105+
106+
print('\t# Make sure your MINERL_DATA_ROOT is set!')
107+
print("\texport MINERL_DATA_ROOT='/your/local/path'")
108+
print("\t# Visualizes a specific trajectory. {}...".format(traj_name[:17]))
109+
print("\tpython3 -m minerl.viewer MineRLTreechop-v0 \\")
110+
print("\t\t{}".format(traj_name))

docs/source/tutorials/getting_started.rst renamed to docs/source/tutorials/index.rst

File renamed without changes.

‎minerl/__init__.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,5 @@
1+
2+
import minerl.dependencies
13
import minerl.data
2-
import minerl.env
4+
import minerl.env
5+
import minerl.env.spaces as spaces

‎minerl/data/__init__.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22
from minerl.data.download import download
33
import os
44

5+
from minerl.data.version import DATA_VERSION, FILE_PREFIX, VERSION_FILE_NAME
6+
7+
import minerl.data.version
58

69
def make(environment=None , data_dir=None, num_workers=4, worker_batch_size=32, minimum_size_to_dequeue=32, force_download=False):
710
"""
@@ -35,6 +38,9 @@ def make(environment=None , data_dir=None, num_workers=4, worker_batch_size=32,
3538
raise ValueError("No data_dir provided and $MINERL_DATA_ROOT undefined."
3639
"Specify force_download=True to download default dataset")
3740

41+
42+
minerl.data.version.assert_version(data_dir)
43+
3844
d = DataPipeline(
3945
os.path.join(data_dir, environment),
4046
environment,

0 commit comments

Comments
 (0)