frequent panic when setting lower redemption_time

[yujonglee]

Default redemption_time is 600ms. When I set it to 300ms, I got frequent panic:

silero-rs/src/lib.rs

Lines 443 to 446 in 26a6460

	None => panic!(
	"Duration {}ms is outside of session audio range",
	duration.as_millis()
	),

Any thoughts?
I think we can start adding different VadConfig when running the test.

Thank you.

[xd009642]

I've done a PR that might fix the issue for you, part of it is that get_speech doesn't take into account the cached speech (and we don't store durations for that which we should continue). But I've made get_current_speech more lenient in the meantime.

If you could try out this PR and let me know if it helps solve your issue: #56

[yujonglee]

@xd009642

thanks for the quick followup. I tried that branch, but unfortunately, problem persists.
Although I feel like it is less frequent.

Seems like you already know the root cause of this - but will it be helpful to give you the audio I used? (for me, it always happen for any audio when I set redemption_time 300ms. so not sure if it will be helpful :) )

fyi - my code using silero.

https://github.com/fastrepl/hyprnote/blob/9e918c87bc5098ebd4b713efb1a4c94ba658c47f/crates/chunker/src/lib.rs#L72-L95

[xd009642]

Yeah if you could give me the audio and a MRE just a main.rs file I can run to recreate it. Looking at that code I just see process called and the transitions handled so not immediately sure where the panic is being hit.

In the meantime if you could give me a backtrace done by setting the env var RUST_BACKTRACE=1 when you run it I can see what's calling the function. Because right now the one place it's called in process looks completely fine to me.

[xd009642]

Oh I might have found it, I've just pushed another commit.

Here's what I think happened. The post speech padding exceeds the session audio length so it experienced an out of bounds error and panicked. I've pushed a commit to the PR where if the padding extends beyond the buffer it truncates the padding space.

Internally, we set the post speech padding equal to the redemption time so I might discuss changing the behaviour to simplify the config in favour of that.

[yujonglee]

thread 'tokio-runtime-worker' panicked at /Users/yujonglee/.cargo/git/checkouts/silero-rs-16a8cd672fe824c4/29fd841/src/lib.rs:450:21:
Duration 2430ms is outside of session audio range
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/core/src/panicking.rs:75:14
   2: silero::VadSession::get_speech
   3: silero::VadSession::process
   4: <chunker::VadChunkStream<S> as futures_core::stream::Stream>::poll_next
             at /Users/yujonglee/dev/hyprnote/crates/chunker/src/lib.rs:124:53
   5: <futures_util::stream::stream::take_while::TakeWhile<St,Fut,F> as futures_core::stream::Stream>::poll_next
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/stream/take_while.rs:80:47
   6: <futures_util::stream::stream::filter_map::FilterMap<St,Fut,F> as futures_core::stream::Stream>::poll_next
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/stream/filter_map.rs:79:47
   7: futures_util::stream::select_with_strategy::poll_side
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/select_with_strategy.rs:220:28
   8: futures_util::stream::select_with_strategy::poll_inner
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/select_with_strategy.rs:234:28
   9: <futures_util::stream::select_with_strategy::SelectWithStrategy<St1,St2,Clos,State> as futures_core::stream::Stream>::poll_next
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/select_with_strategy.rs:270:17
  10: <futures_util::stream::select::Select<St1,St2> as futures_core::stream::Stream>::poll_next
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/select.rs:115:9
  11: futures_util::stream::stream::StreamExt::poll_next_unpin
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/stream/mod.rs:1638:9
  12: <whisper_local::stream::TranscriptionTask<S,whisper_local::stream::MetadataAudioChunkMarker> as futures_core::stream::Stream>::poll_next
             at /Users/yujonglee/dev/hyprnote/crates/whisper-local/src/stream.rs:154:19
  13: futures_util::stream::stream::StreamExt::poll_next_unpin
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/stream/mod.rs:1638:9
  14: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/stream/next.rs:32:9
  15: tauri_plugin_local_stt::server::process_transcription_stream::{{closure}}::{{closure}}
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/macros/select.rs:708:49
  16: <core::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
             at /Users/yujonglee/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/future/poll_fn.rs:151:9
  17: tauri_plugin_local_stt::server::process_transcription_stream::{{closure}}
             at /Users/yujonglee/dev/hyprnote/plugins/local-stt/src/server.rs:205:9
  18: tauri_plugin_local_stt::server::websocket_dual_channel::{{closure}}
             at /Users/yujonglee/dev/hyprnote/plugins/local-stt/src/server.rs:196:60
  19: tauri_plugin_local_stt::server::websocket_with_model::{{closure}}
             at /Users/yujonglee/dev/hyprnote/plugins/local-stt/src/server.rs:149:74
  20: tauri_plugin_local_stt::server::listen::{{closure}}::{{closure}}::{{closure}}
             at /Users/yujonglee/dev/hyprnote/plugins/local-stt/src/server.rs:115:60
  21: axum::extract::ws::WebSocketUpgrade<F>::on_upgrade::{{closure}}
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/axum-0.8.4/src/extract/ws.rs:347:30
  22: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /Users/yujonglee/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9
  23: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/runtime/task/core.rs:365:17
  24: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/loom/std/unsafe_cell.rs:16:9
  25: tokio::runtime::task::core::Core<T,S>::poll
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/runtime/task/core.rs:354:13
  26: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/runtime/task/harness.rs:535:19
  27: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /Users/yujonglee/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:272:9
  28: std::panicking::try::do_call
             at /Users/yujonglee/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:589:40
  29: ___rust_try
  30: std::panicking::try
             at /Users/yujonglee/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:552:19
  31: std::panic::catch_unwind
             at /Users/yujonglee/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:359:14
  32: tokio::runtime::task::harness::poll_future
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/runtime/task/harness.rs:523:18
  33: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/runtime/task/harness.rs:210:27
  34: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/runtime/task/harness.rs:155:15
  35: tokio::runtime::task::raw::poll
             at /Users/yujonglee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.46.1/src/runtime/task/raw.rs:325:5
  36: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
  37: tokio::runtime::scheduler::multi_thread::worker::Context::run
  38: tokio::runtime::context::scoped::Scoped<T>::set
  39: tokio::runtime::context::set_scheduler
  40: tokio::runtime::context::runtime::enter_runtime
  41: tokio::runtime::scheduler::multi_thread::worker::run
  42: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
  43: tokio::runtime::task::core::Core<T,S>::poll
  44: tokio::runtime::task::harness::poll_future
  45: tokio::runtime::task::harness::Harness<T,S>::poll_inner
  46: tokio::runtime::task::raw::poll
  47: tokio::runtime::blocking::pool::Inner::run

[xd009642]

Is that with the latest commit I pushed to the PR or the previous commit? I would think the latest commit on the PR would fix it

Also can you do backtraces with debug mode,, I can see some functions are inlined and therefore removed from the trace and line numbers are missing 👀

[yujonglee]

@xd009642 I think I figured it out.

The problem is pre_speech_pad. VadConfig::default set 600ms for both pre_speech_pad and redemption_time. (and 0 for post_speech_pad, which I was aware of. but not pre_speech_pad).

So If I only set redemption_time to 300ms, it cause trouble, so my fix is:

        let config = VadConfig {
            redemption_time,
            pre_speech_pad: redemption_time,
            post_speech_pad: Duration::from_millis(0),
            min_speech_time: Duration::from_millis(50),
            ..Default::default()
        };

(this works with main too)

[yujonglee]

Is that with the latest commit I pushed to the PR or the previous commit?

Yeah it was :(