Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Anyone know how to set per_process_gpu_memory_fraction ? #3795

Open
austinksmith opened this issue May 16, 2024 · 1 comment
Open

Anyone know how to set per_process_gpu_memory_fraction ? #3795

austinksmith opened this issue May 16, 2024 · 1 comment

Comments

@austinksmith
Copy link

austinksmith commented May 16, 2024

I want to set the following configuration option for tensorflow, I forked this repo and I can see that gpu options are being set on transcribe.py this is fine however... how do I compile from source if I modify these settings? I want to be able to run 2 processes concurrently , i have enough vram if i allocate each one to use 8gb of vram max.



# Assume that you have 12GB of GPU memory and want to allocate ~4GB:
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333)

sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))


I was able to find this discourse forum, https://discourse.mozilla.org/t/how-to-restrict-transcribe-py-from-consuming-whole-gpu-memory/75555/5

this gives the code i need, but how do I compile from source once I change the code?

@austinksmith
Copy link
Author

I figured this out by following a combination of the old docs for version 0.74 and the read the docs version, I had to modify tensorflow directly. Modifying deepspeech to accept this flag didn't actually work. I have a compiled version that limits GPU utilization to 25% of available vram, I tested with a gtx1050TI 4GB restricted to only 1GB and it was actually fairly quick. faster than using 4gb? I think its because when i compiled natively it used some AVX512 cpu extensions to speed things up.

If anyone wants a copy of the modified deepspeech binary, let me know and i'll link it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant