bentoml serve GPU Memory usage #4107
Unanswered
bruce99kang
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
While using bentoml serving an onnx model.

Using code below only, gpu mem usage was around 1500MB.
But, when I built this by
It seemed to be using all of my GPUs instead of using 1 GPU.
Even if I specified CUDA_VISIBLE_DEVICES=1, the GPU Mem usage took around 6000GB for a GPU.
Are there any files needs to be modified?
Beta Was this translation helpful? Give feedback.
All reactions