Skip to content

Latency improvements with the new manual VAD algorithm #368

@clemlesne

Description

@clemlesne

Done:

  • Implement monitoring metrics to follow improvements

Todo:

  • Parallelize more TTS and database calls (study OTEL traces for opportunity confirmation)
  • Reduce dependency calls before sending call to the LLM or defer them
  • Compress the prompt (LLMlingua?)
  • Use a LLM with a lower latency (Phi 4?)
  • Trace the code executions with local debugger to pin points unseen optimizations

Sub-issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions