Bounding boxes from custom detection model are too large (extra padding)

[JoyceFayek]

After fine-tuning the db_resnet50 detection model on my dataset, I tested the model on some images. The predicted bounding boxes around text regions are significantly larger than the actual text—they appear to have extra padding or margin included.

For context, my dataset annotations are tight around the text (no extra margin or padding in the ground truth boxes)

[felixdittrich92]

Hi @JoyceFayek 👋,

Could youz share a few more information about:

• dataset size
• training metrics
• training command

?

Best,
Felix

[JoyceFayek]

• dataset size

  • Validation set (41 samples in 21 batches)
  • Train set (163 samples in 81 batches)

• training metrics

  • (Recall: 25.40% | Precision: 25.99% | Mean IoU: 42.00%)

• training command

  • python3 references/detection/train_pytorch.py db_resnet50 --train_path ............/train --val_path .........../val --pretrained --wb --name experiment3 --save-interval-epoch --output_dir ...../exp3

[felixdittrich92]

Hmm, that doesn't surprise me at this point, since the metrics are really not good.
Maybe try another model for example with linknet_resnet18 I have reached really good results with a low resource dataset at comparable size or db_mobilenet_v3_large.

[JoyceFayek]

I tried linknet_resnet18 and this is the best metric I've got so far (Recall: 59.14% | Precision: 59.21% | Mean IoU: 52.00%)
but still have the same issue, all the text is detected but the bounding boxes are big

[felixdittrich92]

Looks better but still not good 😅

You can try the following:

Add a small modification to the training script

    # Load doctr model
    model = detection.__dict__[args.arch](
        pretrained=args.pretrained,
        assume_straight_pages=not args.rotation,
        class_names=val_set.class_names,
    )
    # Target building params
    model.shrink_ratio = 0.4
    # Postprocessor parameters
    model.postprocessor.unclip_ratio = 1.0

Then retrain

For inference you need to override the unclip_ratio with the same value used for training:

• Additional you can play around a bit with bin_thresh (default is already 0.1 so to detect more choose a lower value 0.5 for example otherwise for possible less but more accurate results a higher score with a max value of 0.99)

    det_model = linknet_resnet18(pretrained=False, pretrained_backbone=False)
    det_params = torch.load(args.model_path, map_location="cpu")
    det_model.load_state_dict(det_params)

    # Postprocessor parameters
    det_model.postprocessor.bin_thresh = args.bin_thresh
    # NOTE: This is important because the current model was trained with this parameter and the current docTR version defaults to 1.5
    det_model.postprocessor.unclip_ratio = 1.0