DEPRECATED

This code is now available (as it was before) at github.com/sheim/vibly/, and you should not download this repo. We used git-lfs for tracking binaries of data, which turned out to be a painful and annoying experience. The new repo has been cleaned out (including the git history), please use that one. The difference is that datasets are now not included when you download the repo, but these can either be regenerated very quickly (couple of minutes on a laptop), or for the RSBL paper, is available on Dryad.

Vibly: viability in state-action space

A Python framework for working with viable sets in state-action space, and can be used for safe learning and for evaluating the effect of morphology on robustness and stability. This code accompanies the following papers:

1. A Learnable Safety Measure Steve Heim, Alexander von Rohr, Sebastian Trimpe, Alexander Badri-Spröwitz, Conference on Robot Learning 2019. Paper-specific examples in demos/measure_learning/.

2. Beyond Basins of Attraction: Quantifying Robustness of Natural Dynamics Steve Heim and Alexander Badri-Spröwitz IEEE Transaction on Robotics 2019. Note: This code implements the core algorithm for the paper, but does not reproduce all results and figures. The Arxiv version contains fewer typos and is recommended.

3. A Little Damping Goes a Long Way: a simulation study of how damping influences task-level stability in running Steve Heim, Matthew Millard, Charlotte Le Mouel, Alexander Badri-Spröwitz Royal Society Biology Letters 2020. Paper-specific examples in demos/damping_study/.

Installation

We recommend using a virtual environment. Note, GPy is only required for the safe learning examples, and can be safely removed from the requirements. To install from your terminal with

PyEnv + PipEnv (recommended):
pipenv install -r requirements.txt
pipenv install -e .

pip:
pip install -r requirements.txt
pip install -e .

conda (not extensively tested):
conda env create -f vibly.yml

You should now be able to run all examples in demos/, and import packages as following:

# import brute-force viability tools
import viability as vibly
# import a model of a dynamical systems
from models import slip
# import classes for active sampling and learning
from measure import active_sampling

Use for viability

Examples are shown in the demos folder. We recommend starting with slip_demo.py, and then computeQ_slip.py.
The viability package contains:

• compute_Q_map: a utility to compute a gridded transition map for N-dimensional systems. Note, this can be computationally intensives (it is essentially brute-forcing an N-dimensional problem). It typically works reasonably well for up to ~4 dimensions.
• parcompute_Q_map: same as above, but parallelized. You typically want to use this, unless running a debugger.
• compute_QV: computes the viability kernel and viable set to within conservative discrete approximation, using the grid generated by compute_Q_map.
• get_feasibility_mask: this can be used to exclude parts of the grid which are infeasible (i.e. are not physically meaningful)
• project_Q2S: Apply an operator (default is an orthogonal projection) from state-action space to state space. Used to compute measures.
• map_S2Q: maps values of each state to state-action space. Used for mapping measures from state space to state-action space.

Create your own dynamics

You can easily add your own models in the models package. The viability code does expect a few helper functions which need to be implemented as well. We recommend using the hovership.py example as a template, as it is simple and heavily commented. For examples of simulating a more complex system, see the slip.py model.

Use for Safe Learning

Select and run an experiment by running run_learning_examples.py in demos/measure_learning/. The experiment details, including algorithm hyper-parameters and initialization, are defined in the experiment file, e.g. demos/measure_learning/hovership_default.py.
The learning algorithm is split between measure/active_sampling, which includes sampling strategy, and measure/estimate_measure, which handles everything dealing with the measure in different spaces. If you're looking to use the measure for a different learning approach, you probably want to look into the active_sampling.py file.

Contact

Feel free to open an issue, or get in touch via [email protected] for any questions, comments, etc.