Does models support GPU?

[IbratDO]

Hello, I have noticed that the default models run in CPU only. So, I was wondering if there is a way to run them in GPU as well?

[freddyaboulton]

Hi @IbratDO ! Sorry I was away on holiday. Right now the default models only run on CPU, correct. You can run your own versions of these models on gpu: https://fastrtc.org/text_to_speech_gallery/#how-to-add-your-own-text-to-speech-model

In theory it would be possible to make it so that fastrtc[tts-gpu] installs onnxruntime-gpu and have kokoro use GPU. LMK if you're interested in adding that!

[IbratDO]

hello, thank you for your response. I did install the fastrtc[tts, stt, vad] as provided on your website. it worked a little slow so I tried to run the KokoroTTS on GPU using onnxruntime-gpu, but it did not work. so do I need to install fastrtc[tts-gpu]?