Meshroom Research

Meshroom's innovation hub for experimenting and nurturing new ideas.

Meshroom-Research comprises a collection of plugins for Meshroom, an intuitive, open-source 3D Reconstruction Software characterized by its nodal UI and architecture. The primary focus of Meshroom-Research lies in seamlessly integrating, testing, and benchmarking various methods within Meshroom. Implemented in Python, it manages dependencies using Conda.

Features

Support for State-of-the-Art Datasets

Meshroom-Research facilitates the exploration of ground truth data from several prominent datasets:

• ETH3D
• DTU
• BlendedMVG
• Nerf synthetic
• ALab synthetic dataset
• Tanks and Temples (Upcoming)
• Skoltech3D (Upcoming)

Simply drag and drop images from the desired dataset into Meshroom. Then, add the LoadDataset node to the camera init's output and select the dataset type. From there, leverage Meshroom's visualization capabilities and connect nodes to the outputs.

load_dataset_demo_3.mp4

3D Reconstruction Method Nodes

Meshroom-Research seamlessly integrates various 3D reconstruction methods into distinct node pipelines, including:

Classical photogrammetry:

• Meshroom
• COLMAP

Deep-learning-based depth map estimation:

• VIZ-mvsnet

Optimization-based via NerfStudio (Upcoming):

• Instant-ngp
• 3D Gaussian Splatting

Benchmarking

Evaluate different stages of the photogrammetry pipeline (SfM, depth map estimation, and meshing) with Meshroom-Research's benchmarking capabilities. Utilize nodes such as CalibrationComparison, DepthMapComparison, and MeshComparison to assess performance. You can run benchmarks via the command line interface using the provided Meshroom project.

# Example usage
python -m benchmark run [OPTIONS] DATASET_PATH

Aggregate results with:

# Example usage
python -m benchmark report [OPTIONS] COMPUTED_OUTPUTS_PATH

Customize your evaluation pipeline with the available evaluation nodes.

3rd Party Imports and Exports

Meshroom-Research facilitates data import and export with various software:

• COLMAP
  • Import SfM calibration
  • Import computed depth
  • Export SfM Calibration
• RealityCapture
  • Import SfM calibration
  • Export SfM Calibration
• Metashape (Upcoming)

New Nodes

Enhance your 3D reconstruction workflow with additional nodes provided by Meshroom-Research. Refer to the nodes directory for a comprehensive list.

Installation

We strongly recommend working within a virtual environment (e.g., Conda).

1. Install Meshroom and AliceVision by following this procedure.
2. Install openimageIO and its Python bindings, e.g., from Conda.
3. Clone and install Meshroom-Research with pip:

git clone https://github.com/alicevision/MeshroomResearch.git
pip install -e ./MeshroomResearch

Developer Quickstart

Contributions to Meshroom-Research are welcomed! Here's a quick overview of the project structure:

• mrrs/core: Basic IOs, utilities, and common geometrical functions.
• mrrs/pipeline: Meshroom pipeline files.
• mrrs/scripts: Scripts, including benchmarking tools.
• mrrs/nodes: Interface nodes for integration into Meshroom.
• mrrs/<feature>: Code related to specific features.

Utilize Meshroom's nodal UI for seamless integration, and refer to the Meshroom's repo for creating custom nodes. We've introduced a new type of node, CondaNode, which automates Conda environment management for your convenience.