Query: multispeaker and multi language in same voice file

[aj7tesh]

@snakers4
First of all I congratulate you and team on doing this wonderful work.
I have a query if you can help.
I have a voice data which is in the below format,
Speaker1-language1 then Speaker1-language2 then Speaker2-language1
All in the same voice file.
I basically want to generate transcripts for this . Could you guide me the most optimum way to do it.
Or is there a way I can split the voice between speaker1 and speaker 2, and then different languages of Speaker1.

Thank you in advance

[snakers4]

Hi,

VAD just detects speech.
If there are pauses you can split utterances and then proceed on utterance level, i.e. apply speaker encoders, language classifiers, STT, etc.
If the speech is continuous, this becomes much more custom and complex.

[aj7tesh]

Got it,
in case of continuous speech is there a way to split it language wise. Something on that?

[snakers4]

Well, you can apply a speaker encoder or a language classifier in a rolling-window fashion (or make it efficient by making it small, or having a small FFN network and a small RNN network applied left-to-right), but that may be very custom.