OpenMiDaS utilizes Gabriel, a platform originally designed for wearable cognitive assistance applications, to stream image data to the backend server which runs an cognitive engines to perform monocular depth estimation. The resulting colormapped images are sent back to the device.
Copyright © 2023 Carnegie Mellon University
This is a developing project.
Unless otherwise stated in the table below, all source code and documentation are under the GPL, Version 2.0. A copy of this license is reproduced in the LICENSE file.
OpenMiDaS uses the MiDaS models from pyTorch hub for monocular depth estimation. It has been tested on Ubuntu 20.04 LTS (focal). It requires an nVidia GPU.
OpenMiDaS has an Android client that is available on the Google PlayStore. It requires an Android device running Android 7.0+ (API level 33).
The quickest way to set up an OpenMiDaS server is to download and run our pre-built Docker container. All of the following steps must be executed as root. We tested these steps using Docker 19.03.
If you do not already have Docker installed, install it using the steps in this Docker install guide.
You can use the CUDA Download page to learn how to install the driver.
If you think you may already have an NVIDIA driver installed, run nvidia-smi. The Driver version will be listed at the top of the table that gets printed.
3. Install the NVIDIA Container Toolkit
Follow these instructions.
docker pull cmusatyalab/openmidas:stableIn the ~/openmidas/server/ directory, there is a template.env file that can be used as an example docker-compose environment. Copy it to .env and then modify it to control things such as the MiDaS model to use.
cd ~/openmidas/server/
cp template.env .env
#edit .env file as necessaryTo launch all the containers and interleave the output from each container in the terminal:
cd ~/openmidas/server
docker-compose upIf you wish to launch the containers in the background, you can pass the -d flag to docker compose. You can then use docker logs to inspect what is happening to individual containers.
cd ~/openmidas/server
docker-compose up -dHitting CTRL-C while docker-compose up is running will stop the containers. However to explicitly destroy them, you can use docker-compose down. This will also destroy the networks.
You can download the client from the Google Play Store.
Alternatively, you can build the client yourself using Android Studio. The source code for the client is located in the android-client directory. You should use the standardDebug build variant.
Servers can be added by entering a server name and address and pressing the + sign button. Once a server has been added, pressing the 'Connect' button will connect to the OpenMiDaS server at that address. Pressing the trash can button will remove the server from the server list.
- Show Screenshot/Recording Buttons - This will enable icons to allow you to capture video or screenshots while running OpenMiDaS.
- Display Metrics - Enabling this option will show the metrics including FPS.
- Resolution - Configure the resoultion to capture at. This will have a moderate impact in the computation time on the server.
- Gabriel Token Limit - Allows configuration of the token-based flow control mechanism in the Gabriel platform. This indicates how many frames can be in flight before a response frame is received back from the server. The minimum of the token limit specified in the app and the number of tokens specified on the server will be used.
- MiDaS Model - Configure the model used on the server. MiDaS has 3 pretrained models: Large, Hybrid, and Small.
- Colormap - Configure which colormap to apply to the depth estimation. These correspond to OpenCV's colormaps.
Once connected to a server, an icon is displayed in the upper right hand corner which allows one to toggle between front- and rear-facing cameras.
Please see the CREDITS file for a list of acknowledgments.