3D Scene Understanding for Autonomous Driving

A comprehensive pipeline for processing KITTI dataset images through preprocessing, 3D reconstruction, object detection, semantic segmentation, and evaluation.

Overview

This project implements a complete pipeline for analyzing KITTI autonomous driving data, including:

• Image preprocessing and standardization
• 3D scene reconstruction using COLMAP
• Object detection using YOLOv8
• Semantic segmentation
• Pipeline evaluation and visualization

Prerequisites

Dependencies

pip install -r requirements.txt

Required packages:

• OpenCV
• NumPy
• COLMAP
• Ultralytics (YOLOv8)
• Open3D
• Matplotlib
• tqdm

Dataset

Download the KITTI dataset from the official website.

Pipeline Components

1. Data Preprocessing

• Standardizes image sizes
• Organizes data structure
• Preserves timestamp information

2. 3D Reconstruction

• Feature extraction and matching
• Sparse reconstruction
• Dense reconstruction
• Camera pose estimation

3. Object Detection

• YOLOv8-based detection
• Multiple object class support
• Confidence-based filtering

4. Semantic Segmentation

• Per-pixel semantic labeling
• Multi-class segmentation
• Instance segmentation support

5. Evaluation

• Detection accuracy metrics
• Segmentation quality assessment
• Visualization generation
• Comprehensive reporting

Usage

1. Preprocess the data:

python preprocess.py --data_root data --sequence 2011_09_26_drive_0009_sync

2. Run 3D reconstruction:

python reconstruct.py --input preprocessed_data --output reconstruction_output

3. Perform object detection:

python detect.py --input preprocessed_data --output detection_output

4. Run semantic segmentation:

python segment.py --input preprocessed_data --output segmentation_output

5. Generate evaluation report:

python evaluate.py --base_path .

License

This project is licensed under the MIT License.

Acknowledgments

• KITTI Dataset Link
• COLMAP Structure-from-Motion Link
• YOLOv8 by Ultralytics Link