GOTPR: General Outdoor Text-based Place Recognition Using Scene Graph Retrieval with OpenStreetMap

[Project 🗃️], [Video 📺], [Paper 📕], [arXiv 📖]

Donghwi Jung, Keonwoo Kim, Seong-Woo Kim

We introduce GOTPR, a place recognition approach that employs scene graphs to minimize storage requirements and utilizes candidate extraction to enhance processing speed. By utilizing OpenStreetMap data, GOTPR facilitates efficient outdoor text-based place recognition without the need for pre-generated maps, ensuring high accuracy and real-time performance for robotic systems.

Setup

Setting an environments

conda env create -f environment.yml
conda activate GOTPR

Install the NLTK

Install the trained pipeline for word2vec embeddings.

pip install spacy
python -m spacy download en_core_web_lg

Install the vectorDB (Milvus)

The vectorDB is required for extracting scene graph candidates. To install it, follow the command below or instructions at link. If you're unable to install the vectorDB for any reason, you can skip this step and continue with the code without the candidate extraction by setting the use_candidates_extraction in config.py as False. However, skipping the candidates extraction step will slow down the process.

pip install -U pymilvus

Setup directories

Create data related directories for running the code.

bash setup_directories.sh

Dataset

• Download scene graphs and model checkpoints. The scene graphs and model checkpoints were created using GPS coordinates derived from the East-North-Up (ENU) coordinates of the KITTI360Pose dataset.
• Move the downloaded data to the data directory.
• Unzip the downloaded data.

Train

To modify the training arguments, refer to the config.py file.

python train.py

Evaluation

To modify the evaluation arguments, consult the config.py file. Additionally, if you have not installed vectorDB (Milvus) as outlined in Install the vectorDB (Milvus), set this value to False.

python eval.py

Visualization

Select a text scene graph to perform scene retrieval, then visualize the matching text and OSM scene graphs. The index of the text scene graph to be checked can be specified using the --visualization_graph_index argument when executing the Python script.

python visualize_scene_retrieval_results.py --visualization_graph_index <text_graph_index>
# e.g. python visualize_scene_retrieval_results.py ---visualization_graph_index 2500 

Citation

@article{jung2025gotpr,
  title={GOTPR: General Outdoor Text-based Place Recognition Using Scene Graph Retrieval with OpenStreetMap},
  author={Jung, Donghwi and Kim, Keonwoo and Kim, Seong-Woo},
  journal={IEEE Robotics and Automation Letters},
  year={2025},
  publisher={IEEE}
}

Acknowledgements

The codes and datasets in this repository are based on Where am I?, Milvus, and Text2Pos.

License

Copyright 2025, Donghwi Jung, Keonwoo Kim, Seong-Woo Kim, Autonomous Robot Intelligence Lab, Seoul National University.

This project is free software made available under the MIT License. For details see the LICENSE file.