Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Audio Playback Stops Randomly/Sturrers Until Manually Pressing the Red "Stop" Button #4

Open
GCPins opened this issue Mar 19, 2023 · 2 comments
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@GCPins
Copy link

GCPins commented Mar 19, 2023

Describe the bug
The audio playback of the synthesized/cloned voice stops randomly and stutters.

To Reproduce
Steps to reproduce the behavior:

  1. Start RTVC
  2. Speak and wait for the program to transcribe your message
  3. Wait for the program to send the transcribed audio to ElevenLabs API
  4. The audio playback will start then immediately stop (usually after the first word) and will not continue (or will stutter) until you press the red "Stop" button.

This happens regardless of whether or not you continue to speak while the program is listening for speech.

Expected behavior
No stuttering or halting of audio playback.

Smartphone (please complete the following information):

  • Device: Samsung S21 FE
  • OS: Android 13
  • Browser: Chrome (latest)

Additional context

@GCPins
Copy link
Author

GCPins commented Mar 19, 2023

Uodate: everything works as expected on my desktop (Windows 11/Chrome), so this might be an issues on my end.

@hparcells
Copy link
Owner

I have noticed this behavior throughout the development of this project, though in the final (current) version I no longer notice this happen (at least on desktop).

I suspect this is due to one of two issues:

  1. The looping nature of the logic: that is, when mic input stops, listening stops, makes a request, and plays the resulting audio, ending by starting back up at the top. Therefore, it might have to do something with the JavaScript engine, which could be the second issue...
  2. The JavaScript engine: Something weird could be happening on mobile browsers (slower execution?). This project uses the Web Speech API, which is supposedly supported on most browsers (except Firefox). The solution could be to use an actual speech-to-text transcription service, but this would increase the cost of running the project and more importantly increase the processing time from speech to speech.

I will look into this more (including testing on mobile devices) when I get a chance.

@hparcells hparcells added bug Something isn't working help wanted Extra attention is needed labels Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants