-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Open
Description
- have a system with an nvidia gpu. the server in
examples
builds with cuda support, no issue. - build the go examples with
GGML_CUDA=1 make example
and observe ggml-cuda being built. - run go-whisper with
build_go/go-whisper -model ../../models/ggml-medium.bin samples/jfk.wav
and observe that it actually didn't build or link against cuda:
$ build_go/go-whisper -model ../../models/ggml-medium.bin samples/jfk.wav
whisper_init_from_file_with_params_no_state: loading model from '../../models/ggml-medium.bin'
whisper_init_with_params_no_state: use gpu = 1
whisper_init_with_params_no_state: flash attn = 0
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw = 0
whisper_init_with_params_no_state: devices = 1
whisper_init_with_params_no_state: backends = 1
whisper_model_load: loading model
whisper_model_load: n_vocab = 51865
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 1024
whisper_model_load: n_audio_head = 16
whisper_model_load: n_audio_layer = 24
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 1024
whisper_model_load: n_text_head = 16
whisper_model_load: n_text_layer = 24
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 4 (medium)
whisper_model_load: adding 1608 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: CPU total size = 1533.14 MB
whisper_model_load: model size = 1533.14 MB
whisper_backend_init_gpu: no GPU found
whisper_init_state: kv self size = 50.33 MB
whisper_init_state: kv cross size = 150.99 MB
whisper_init_state: kv pad size = 6.29 MB
whisper_init_state: compute buffer (conv) = 28.57 MB
whisper_init_state: compute buffer (encode) = 170.17 MB
whisper_init_state: compute buffer (cross) = 7.73 MB
whisper_init_state: compute buffer (decode) = 98.20 MB
system_info: n_threads = 20 / 20 | WHISPER : COREML = 0 | OPENVINO = 0 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | OPENMP = 1 | REPACK = 1 |
Metadata
Metadata
Assignees
Labels
No labels