How to adjust threshold for detections?

[robmarkcole]

The vessel detections are thresholded at 0.8 but I would like to review lower confidence predictions. The only reference I can find in the codebase is in the training config - can it be adjusted in the prediction pipeline? I've been using the docker API
Thanks

[favyen2]

I think you would need to change it in the model config YAML. You can change it to a low value there, and then threshold the detections afterward since the confidence score appears in the JSON/GeoJSON output.

We can keep the issue open to see if we want to add a parameter for it.

[robmarkcole]

@favyen2 I don't suppose you mapped the precision-recall curve?

[favyen2]

I think if you run model test it should show precision/recall numbers at different thresholds, here it is in a table:

Threshold	Precision	Recall
0.1	52%	93%
0.2	58%	91%
0.3	62%	89%
0.4	65%	88%
0.5	68%	86%
0.6	70%	84%
0.7	74%	82%
0.8	77%	79%
0.9	82%	74%

[robmarkcole]

OK here is what I'm doing successfully:

• Edit threshold in rslearn_projects/data/sentinel2_vessels/config.yaml
• Now run a modified command:

docker run --rm -p 5556:5556 \
    -e RSLP_PREFIX=/project_data \
    -e SENTINEL2_PORT=5556 \
    -v $PWD/project_data:/project_data \
    -v $PWD/data/sentinel2_vessels/config.yaml:/opt/rslearn_projects/data/sentinel2_vessels/config.yaml:ro \
    --shm-size=15g \
    --gpus all \
    ghcr.io/allenai/sentinel2-vessel-detection:sentinel2_vessels_v0.0.4

I'm finding that many of the lower confidence predictions are usable