All the output pt files from custom training has same size

I tried to custom training from the google colab which is working but all the output pt files has same size of 41.3 MB irrespective of number of images I train, and I suspect that is the reason of low accuracy compared to pretrained models for object detection.
Can someone mention why all the output pt files are same size