Set batchsize in inference

[ManuelM95]

Hello,

thx for the code :).

Is there any way to set a higher batchsize in inference as still 2/3 of my GPU memory is empty - if not would it be easy to add? Looking forward to any help.