Inference result with space after adding scorer

[JRMeyer]

>>> tqjls
[February 23, 2021, 7:03am]

hi, i am facing a thing that everything goes well when i just use the
pbmm model to infer, but after adding a scorer model, there is many
spaces between every chinese word and the result become very weired: slash
Before using a scorer model: slash
root slash 32884f327085:/DeepSpeech slash # deepspeech slash --model model.pbmm slash --audio
1690041315330513107_aaaf20a46fdf11ea_C slash _12146_20200414_090204.wav
slash --beam_width 3 slash
Loading model from file ./model.pbmm slash
TensorFlow: v2.3.0-6-g23ad988 slash
DeepSpeech: v0.9.3-0-gf2e9c85 slash
2021-02-23 06:39:29.321567: I
tensorflow/core/platform/cpu_feature_guard.cc:142 slash ] This TensorFlow
binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to
use the following CPU instructions in performance-critical operations:
AVX2 AVX512F FMA slash
To enable them in other operations, rebuild TensorFlow with the
appropriate compiler flags. slash
Loaded model in 0.0526s. slash
Running inference. slash
呃我家的电视不能看了

but after using a scorer mode，the result become: slash
root slash 32884f327085:/DeepSpeech slash # deepspeech slash --model model.pbmm slash --audio
1690041315330513107_aaaf20a46fdf11ea_C slash _12146_20200414_090204.wav
slash --beam_width 3 slash --scorer word_noprune_alphabet.scorer slash --lm_alpha 0.8
slash --lm_beta 100 slash
Loading model from file ./model.pbmm slash
TensorFlow: v2.3.0-6-g23ad988 slash
DeepSpeech: v0.9.3-0-gf2e9c85 slash
2021-02-23 06:38:24.955914: I
tensorflow/core/platform/cpu_feature_guard.cc:142 slash ] This TensorFlow
binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to
use the following CPU instructions in performance-critical operations:
AVX2 AVX512F FMA slash
To enable them in other operations, rebuild TensorFlow with the
appropriate compiler flags. slash
Loaded model in 0.0527s. slash
Loading scorer from files ./word_noprune_alphabet.scorer slash
Loaded scorer in 0.0455s. slash
Running inference. slash
呃 我 这 家 的 电 视 slash
Inference took 15.492s for 18.120s audio file.

and if we have a small lm_beta: slash
root slash 32884f327085:/DeepSpeech slash # deepspeech slash --model model.pbmm slash --audio
1690041315330513107_aaaf20a46fdf11ea_C slash _12146_20200414_090204.wav
slash --beam_width 3 slash --scorer noprune_alphabet.scorer slash --lm_alpha 0.8
slash --lm_beta 10 slash
Loading model from file ./model.pbmm slash
TensorFlow: v2.3.0-6-g23ad988 slash
DeepSpeech: v0.9.3-0-gf2e9c85 slash
2021-02-23 07:00:54.037949: I
tensorflow/core/platform/cpu_feature_guard.cc:142 slash ] This TensorFlow
binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to
use the following CPU instructions in performance-critical operations:
AVX2 AVX512F FMA slash
To enable them in other operations, rebuild TensorFlow with the
appropriate compiler flags. slash
Loaded model in 0.104s. slash
Loading scorer from files ./noprune_alphabet.scorer slash
Loaded scorer in 0.0519s. slash
Running inference. slash
呃 slash
Inference took 15.654s for 18.120s audio file.

very thankful if anyone can give some advice !!

[This is an archived TTS discussion thread from discourse.mozilla.org/t/inference-result-with-space-after-adding-scorer]

[JRMeyer]

>>> lissyx
[February 23, 2021, 8:11am]

Are you using a Chinese trained model? Self trained or our model ?

[Archived Post]