StreamingTranscriber Authentication Fails in Browser Environment

[xdmarttt]

The StreamingTranscriber class fails to authenticate properly in browser environments, throwing the:

SyntaxError: Failed to construct 'WebSocket': The subprotocol '[object Object]' is invalid.
at factory (/node_modules/.vite/deps/assemblyai.js?v=dc373e06:132:12)

This occurs because the StreamingTranscriber incorrectly passes authentication headers to the WebSocket constructor, which is not supported in browser environments.

This bug completely breaks the StreamingTranscriber functionality in browser environments.

Environment

• SDK Version: Latest TypeScript SDK
• Environment: Browser (client-side)
• Authentication Method: Both API key and temporary token fail
• Error Location: polyfillWebSocketFactory call in connect() method

Root Cause Analysis

The issue stems from inconsistent authentication handling between StreamingTranscriber and RealtimeTranscriber classes:

❌ Current Broken Implementation (StreamingTranscriber)

https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/src/services/streaming/service.ts#L121-L123

✅ Working Implementation (RealtimeTranscriber)

https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/src/services/realtime/service.ts#L185-L205

Key Issues in StreamingTranscriber

1. Browser WebSocket API Incompatibility: The StreamingTranscriber always passes a headers object to polyfillWebSocketFactory(), but browser WebSocket constructors don't support custom headers. The browser WebSocket API signature is:

   new WebSocket(url, protocols?) // protocols is string[] or string, NOT an object

   When {headers: {...}} is passed as the second parameter, browsers interpret it as subprotocols, causing the [object Object] error.

2. Headers passed for token authentication: When using a temporary token, the StreamingTranscriber still tries to pass headers to the WebSocket constructor, which fails in browsers since tokens should be sent via URL parameters instead.

3. Missing token URL parameter: Unlike RealtimeTranscriber, the StreamingTranscriber doesn't add the token to the WebSocket URL as a query parameter in the connectionUrl() method.

4. No browser environment detection: StreamingTranscriber lacks the browser compatibility warning that RealtimeTranscriber provides.

Proposed Fix

1. Update connect() method in StreamingTranscriber:

connect() {
  return new Promise<BeginEvent>((resolve) => {
    if (this.socket) {
      throw new Error("Already connected");
    }

    const url = this.connectionUrl();

    // Handle authentication like RealtimeTranscriber
    if (this.token) {
      this.socket = polyfillWebSocketFactory(url.toString());
    } else {
      if (conditions.browser) {
        console.warn(
          `API key authentication is not supported for StreamingTranscriber in browser environment. Use temporary token authentication instead.
Learn more at https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/docs/compat.md#browser-compatibility.`
        );
      }
      this.socket = polyfillWebSocketFactory(url.toString(), {
        headers: { Authorization: this.apiKey },
      });
    }

    // ... rest of the method
  });
}

2. Update connectionUrl() method in StreamingTranscriber:

private connectionUrl(): URL {
  const url = new URL(this.params.websocketBaseUrl ?? "");

  if (url.protocol !== "wss:") {
    throw new Error("Invalid protocol, must be wss");
  }

  const searchParams = new URLSearchParams();

  // Add token to URL parameters like RealtimeTranscriber
  if (this.token) {
    searchParams.set("token", this.token);
  }

  searchParams.set("sample_rate", this.params.sampleRate.toString());

  // ... rest of existing parameter handling

  url.search = searchParams.toString();
  return url;
}

[nathan-chappell]

Will this be addressed? I tried the proposed fix above, but it didn't quite work. I got this to work though. Main thing is to add the token to the query param, and do all the other bookeeping stuff. (Also, don't accidentally send header where protocol is expected).

// @ts-nocheck
import { BeginEvent, StreamingTranscriber } from 'assemblyai';
import { Future } from '../common/future';

export class _StreamingTranscriber extends StreamingTranscriber {
    // @ts-ignore
    public socket: WebSocket;

    async connect() {
        const beginEvent = new Future<BeginEvent>();
        if (this.socket) {
            throw new Error('Already connected');
        }

        const url: URL = this.connectionUrl();
        url.searchParams.set('token', this.token);

        // Handle authentication like RealtimeTranscriber
        // @ts-ignore
        this.socket = new WebSocket(url.toString());

        this.socket.binaryType = 'arraybuffer';

        this.socket.onopen = () => {
            beginEvent.resolve();
        };

        this.socket.onclose = ({ code, reason }: CloseEvent) => {
            beginEvent.reject();
            if (!reason) {
                if (code in StreamingErrorMessages) {
                    reason = StreamingErrorMessages[code as StreamingErrorTypeCodes];
                }
            }
            this.listeners.close?.(code, reason);
        };

        this.socket.onerror = (event: ErrorEvent) => {
            beginEvent.reject();
            if (event.error) this.listeners.error?.(event.error as Error);
            else this.listeners.error?.(new Error(event.message));
        };

        this.socket.onmessage = ({ data }: MessageEvent) => {
            beginEvent.resolve();
            const message = JSON.parse(data.toString()) as StreamingEventMessage;

            if ('error' in message) {
                this.listeners.error?.(new StreamingError(message.error));
                return;
            }

            switch (message.type) {
                case 'Begin': {
                    beginEvent.resolve(message);
                    this.listeners.open?.(message);
                    break;
                }
                case 'Turn': {
                    this.listeners.turn?.(message);
                    break;
                }
                case 'Termination': {
                    this.sessionTerminatedResolve?.();
                    break;
                }
            }
        };
        await beginEvent;
    }
}

[gsharp-aai]

@xdmarttt @nathan-chappell Hey all! This should now be fixed with the latest release (v4.13.2). Here's a link to the relevant portion of the README.

Please try upgrading to this new version and let me know if you continue to encounter this issue.

[nathan-chappell]

@gsharp-aai It's working, thanks!