Adding support to WebsocketTransport #1911

filipi87 · 2025-05-28T21:38:45Z

Fixes inside the ProtobufSerializer.
A Pipecat example demonstrating how to use WebsocketTransport

filipi87 · 2025-05-28T21:39:53Z

examples/websocket/client/package.json

+  },
+  "dependencies": {
+    "@pipecat-ai/client-js": "^0.3.5",
+    "@pipecat-ai/realtime-websocket-transport": "^0.3.5",


I still need to publish a new version which will include the PipecatWebsocketTransport.
Right now I am using the version from this PR:

Creating a Pipecat websocket transport. pipecat-client-web-transports#42

codecov · 2025-05-28T21:41:13Z

Codecov Report

Attention: Patch coverage is 22.22222% with 7 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/pipecat/serializers/protobuf.py	22.22%	7 Missing ⚠️

Files with missing lines	Coverage Δ
src/pipecat/serializers/protobuf.py	`79.71% <22.22%> (-8.82%)`	⬇️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

markbackman · 2025-05-29T14:33:36Z

In terms of organization for the examples, we already have a websocket-server example. Should this new example replace the current one?

examples/websocket/server/bot.py

examples/websocket/server/env.example

examples/websocket/server/requirements.txt

filipi87 · 2025-05-29T20:56:20Z

In terms of organization for the examples, we already have a websocket-server example. Should this new example replace the current one?

@markbackman, I have removed the websocket-server example now, and changed this one, so when you execute the server you can choose what do you wish to use, 'fast_api' or 'websocket_server'.

WEBSOCKET_SERVER= # Options: 'fast_api' or 'websocket_server'

What do you think ?

markbackman · 2025-05-30T13:02:20Z

WEBSOCKET_SERVER= # Options: 'fast_api' or 'websocket_server'

What do you think ?

This makes sense!

markbackman · 2025-05-30T15:18:11Z

examples/websocket/server/bot_fast_api.py

+        pipeline,
+        params=PipelineParams(
+            allow_interruptions=True,
+            observers=[RTVIObserver(rtvi)],


observers should be a PipelineTask param. It's deprecated as a PipelineParams param.

markbackman · 2025-05-30T15:19:11Z

examples/websocket/server/bot_fast_api.py

+        ),
+    )
+
+    @rtvi.event_handler("on_client_ready")


We do this for simple-chatbot:

@rtvi.event_handler("on_client_ready") async def on_client_ready(rtvi): await rtvi.set_bot_ready() # Kick off the conversation await task.queue_frames([context_aggregator.user().get_context_frame()])

Does this make sense here too?

markbackman · 2025-05-30T15:19:49Z

examples/websocket/server/bot_websocket_server.py

@@ -0,0 +1,110 @@
+#


Same comments as the bot_fast_api.py example.

observer

on_client_ready

GeminiMultimodalLiveLLMService args

markbackman · 2025-05-30T15:20:11Z

examples/websocket/server/bot_fast_api.py

+    llm = GeminiMultimodalLiveLLMService(
+        api_key=os.getenv("GOOGLE_API_KEY"),
+        voice_id="Puck",  # Aoede, Charon, Fenrir, Kore, Puck
+        transcribe_user_audio=True,


This now happens automatically.

Suggested change

transcribe_user_audio=True,

markbackman · 2025-05-30T15:20:24Z

examples/websocket/server/bot_fast_api.py

+        api_key=os.getenv("GOOGLE_API_KEY"),
+        voice_id="Puck",  # Aoede, Charon, Fenrir, Kore, Puck
+        transcribe_user_audio=True,
+        transcribe_model_audio=True,


Same here.

Suggested change

transcribe_model_audio=True,

markbackman

Looks really good! Just a few comments, but overall this is really nice to have.

…t to use.

…o start the bot inside on_client_ready.

filipi87 · 2025-06-06T20:23:54Z

Thank you for the review @markbackman 🎉

filipi87 commented May 28, 2025

View reviewed changes

filipi87 changed the title ~~A Pipecat example demonstrating how to use PipecatWebsocketTransport~~ Adding support to PipecatWebsocketTransport May 29, 2025

filipi87 mentioned this pull request May 29, 2025

Creating a Pipecat websocket transport. pipecat-ai/pipecat-client-web-transports#42

Merged

filipi87 requested review from aconchillo, markbackman and mattieruth May 29, 2025 14:17

markbackman reviewed May 29, 2025

View reviewed changes

examples/websocket/server/bot.py Outdated Show resolved Hide resolved

markbackman reviewed May 29, 2025

View reviewed changes

examples/websocket/server/bot.py Outdated Show resolved Hide resolved

markbackman reviewed May 29, 2025

View reviewed changes

examples/websocket/server/env.example Outdated Show resolved Hide resolved

markbackman reviewed May 29, 2025

View reviewed changes

examples/websocket/server/requirements.txt Outdated Show resolved Hide resolved

filipi87 requested a review from markbackman May 29, 2025 20:56

markbackman reviewed May 30, 2025

View reviewed changes

markbackman approved these changes May 30, 2025

View reviewed changes

filipi87 changed the title ~~Adding support to PipecatWebsocketTransport~~ Adding support to WebsocketTransport Jun 6, 2025

filipi87 added 3 commits June 6, 2025 17:07

Adding support in ProtobufFrameSerializer to deserialize MessageFrame.

0286502

A Pipecat example demonstrating how to use WebsocketTransport

1f51b6e

Removing the old websocket-server example

e9f041e

filipi87 force-pushed the filipi/websocket_transport_example branch from 689b501 to e9f041e Compare June 6, 2025 20:09

filipi87 added 3 commits June 6, 2025 17:12

Improving the readme to mention that can choose which server websocke…

d774a23

…t to use.

Mentioning the new websocket example in the changelog.

8293588

Refactoring to use the observer inside the pipelinetask, and moving t…

1e74476

…o start the bot inside on_client_ready.

filipi87 merged commit c9575ea into main Jun 6, 2025
5 of 6 checks passed

filipi87 deleted the filipi/websocket_transport_example branch June 6, 2025 20:25

Adding support to WebsocketTransport #1911

Adding support to WebsocketTransport #1911

Uh oh!

Conversation

filipi87 commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

markbackman commented May 29, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

filipi87 commented May 29, 2025

Uh oh!

markbackman commented May 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

markbackman May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

markbackman left a comment

Choose a reason for hiding this comment

Uh oh!

filipi87 commented Jun 6, 2025

Uh oh!

Uh oh!

Uh oh!

filipi87 commented May 28, 2025 •

edited

Loading

codecov bot commented May 28, 2025 •

edited

Loading

markbackman May 30, 2025 •

edited

Loading