Skip to content

TextAudioSynchronizer error in JS voice agent pipeline #456

@allanjsx

Description

@allanjsx

Describe the bug

Encountered random crash very often of TextAudioSynchronizer is closed

Relevant log output

waiting for participant
starting assistant example agent for connected-protocol
executing weather function for Singapore
file:///Users/swarm-container-caumyw2/node_modules/.pnpm/@[email protected]_@[email protected]/node_modules/@livekit/agents/dist/transcription.js:215
      throw new Error("TextAudioSynchronizer is closed");
            ^

Error: TextAudioSynchronizer is closed
    at #checkNotClosed (file:///Users/swarm-container-caumyw2/node_modules/.pnpm/@[email protected]_@[email protected]/node_modules/@livekit/agents/dist/transcription.js:215:13)
    at TextAudioSynchronizer.markTextSegmentEnd (file:///Users/swarm-container-caumyw2/node_modules/.pnpm/@[email protected]_@[email protected]/node_modules/@livekit/agents/dist/transcription.js:72:25)
    at file:///Users/swarm-container-caumyw2/node_modules/.pnpm/@[email protected]_@[email protected]/node_modules/@livekit/agents/dist/pipeline/agent_output.js:156:25
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.19.2
libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
[20:04:39.145] WARN (82128): job process exited unexpectedly; this likely means the error above caused a crash
    err: {
      "type": "Error",
      "message": "Channel closed",
      "stack":
          Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
              at target.send (node:internal/child_process:753:16)
              at Timeout._onTimeout (file:///Users/swarm-container-caumyw2/node_modules/.pnpm/@[email protected]_@[email protected]/node_modules/@livekit/agents/dist/ipc/supervised_proc.js:46:17)
              at listOnTimeout (node:internal/timers:581:17)
              at process.processTimers (node:internal/timers:519:7)
      "code": "ERR_IPC_CHANNEL_CLOSED"
    }

Describe your environment

System:
    OS: macOS 15.1.1
    CPU: (8) arm64 Apple M1 Pro
    Memory: 201.84 MB / 32.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.19.2 - ~/.nvm/versions/node/v20.19.2/bin/node
    Yarn: 1.22.22 - ~/.nvm/versions/node/v18.18.2/bin/yarn
    npm: 10.8.2 - ~/.nvm/versions/node/v20.19.2/bin/npm
    pnpm: 8.6.12 - ~/.nvm/versions/node/v18.18.2/bin/pnpm
  npmPackages:
    @livekit/agents: ^0.7.0 => 0.7.0 
    @livekit/agents-plugin-deepgram: ^0.5.3 => 0.5.3 
    @livekit/agents-plugin-elevenlabs: ^0.6.1 => 0.6.1 
    @livekit/agents-plugin-openai: ^0.9.0 => 0.9.0 
    @livekit/agents-plugin-silero: ^0.5.3 => 0.5.3 
    @livekit/rtc-node: ^0.13.3 => 0.13.3 

Minimal reproducible example

Reproduce step:

  1. Create a sandbox
  2. Set up the local environment for the sandbox with Node agent pipleline mode
  3. Talk to the sandbox agent, after a few message it will report this error.

Reproduce video: Screen recording (error after the 3rd voice interaction)

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions