Native two way audio HA support

[starkillerOG]

I am trying to get two way audio supported by the build in go2rtc integration of Home Assistant (default camera player of HA).
That way everyone using Home Assistant can use two way audio and does not have to specifically set it up themselfs using custom cards.

I have it working for Chrome and the HA Android Companion app.
However on Firefox I am getting the following error:
2025-07-07 18:19:27.671 WARNING (MainThread) [homeassistant.components.go2rtc.server] pc ERROR: 2025/07/07 18:19:27 Incoming unhandled RTP ssrc(1894855586), OnTrack will not be fired. mid RTP Extensions required for Simulcast

Do you have any idea how I could resolve this error on Firefox?
I think it might have to do with the renegotiation I am performing because I only change the transceiver direction to "sendrecv" once the user presses the microphone button for the first time (then I start toggeling the .enable = true/false on the microphone track).

My PRs are here:
Frontend: https://github.com/home-assistant/frontend/pull/26091/files
Core: https://github.com/home-assistant/core/pull/148282/files

The SDP offer (after enabeling 2-way audio) on chrome:

WebRTC: start webRtcOffer v=0
o=- 6767500846641700837 3 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 111 63 9 0 8 13 110 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:H5/1
a=ice-pwd:1/lzAgS2kMhgmWpJl4P6+izr
a=ice-options:trickle
a=fingerprint:sha-256 D4:44:90:39:6D:0B:58:9C:1F:D2:6C:26:92:DA:06:37:E5:FA:D0:29:13:FC:55:9C:EE:A6:93:4A:67:9B:AC:1B
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=sendrecv
a=msid:- 74771d38-40ec-4b70-93ba-455b49811e31
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=fmtp:63 111/111
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:126 telephone-event/8000
a=ssrc:2178830615 cname:qP4zwuuQMM1KEzvU
a=ssrc:2178830615 msid:- 74771d38-40ec-4b70-93ba-455b49811e31
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 35 36 37 38 103 104 107 108 109 114 115 116 117 118 39 40 41 42 43 44 45 46 47 48 119 120 121 122 49 50 51 52 123 124 125 53
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:H5/1
a=ice-pwd:1/lzAgS2kMhgmWpJl4P6+izr
a=ice-options:trickle
a=fingerprint:sha-256 D4:44:90:39:6D:0B:58:9C:1F:D2:6C:26:92:DA:06:37:E5:FA:D0:29:13:FC:55:9C:EE:A6:93:4A:67:9B:AC:1B
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:35 VP9/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=fmtp:35 profile-id=1
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:37 VP9/90000
a=rtcp-fb:37 goog-remb
a=rtcp-fb:37 transport-cc
a=rtcp-fb:37 ccm fir
a=rtcp-fb:37 nack
a=rtcp-fb:37 nack pli
a=fmtp:37 profile-id=3
a=rtpmap:38 rtx/90000
a=fmtp:38 apt=37
a=rtpmap:103 H264/90000
a=rtcp-fb:103 goog-remb
a=rtcp-fb:103 transport-cc
a=rtcp-fb:103 ccm fir
a=rtcp-fb:103 nack
a=rtcp-fb:103 nack pli
a=fmtp:103 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:104 rtx/90000
a=fmtp:104 apt=103
a=rtpmap:107 H264/90000
a=rtcp-fb:107 goog-remb
a=rtcp-fb:107 transport-cc
a=rtcp-fb:107 ccm fir
a=rtcp-fb:107 nack
a=rtcp-fb:107 nack pli
a=fmtp:107 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:108 rtx/90000
a=fmtp:108 apt=107
a=rtpmap:109 H264/90000
a=rtcp-fb:109 goog-remb
a=rtcp-fb:109 transport-cc
a=rtcp-fb:109 ccm fir
a=rtcp-fb:109 nack
a=rtcp-fb:109 nack pli
a=fmtp:109 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:114 rtx/90000
a=fmtp:114 apt=109
a=rtpmap:115 H264/90000
a=rtcp-fb:115 goog-remb
a=rtcp-fb:115 transport-cc
a=rtcp-fb:115 ccm fir
a=rtcp-fb:115 nack
a=rtcp-fb:115 nack pli
a=fmtp:115 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:116 rtx/90000
a=fmtp:116 apt=115
a=rtpmap:117 H264/90000
a=rtcp-fb:117 goog-remb
a=rtcp-fb:117 transport-cc
a=rtcp-fb:117 ccm fir
a=rtcp-fb:117 nack
a=rtcp-fb:117 nack pli
a=fmtp:117 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f
a=rtpmap:118 rtx/90000
a=fmtp:118 apt=117
a=rtpmap:39 H264/90000
a=rtcp-fb:39 goog-remb
a=rtcp-fb:39 transport-cc
a=rtcp-fb:39 ccm fir
a=rtcp-fb:39 nack
a=rtcp-fb:39 nack pli
a=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f
a=rtpmap:40 rtx/90000
a=fmtp:40 apt=39
a=rtpmap:41 H264/90000
a=rtcp-fb:41 goog-remb
a=rtcp-fb:41 transport-cc
a=rtcp-fb:41 ccm fir
a=rtcp-fb:41 nack
a=rtcp-fb:41 nack pli
a=fmtp:41 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=f4001f
a=rtpmap:42 rtx/90000
a=fmtp:42 apt=41
a=rtpmap:43 H264/90000
a=rtcp-fb:43 goog-remb
a=rtcp-fb:43 transport-cc
a=rtcp-fb:43 ccm fir
a=rtcp-fb:43 nack
a=rtcp-fb:43 nack pli
a=fmtp:43 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=f4001f
a=rtpmap:44 rtx/90000
a=fmtp:44 apt=43
a=rtpmap:45 AV1/90000
a=rtcp-fb:45 goog-remb
a=rtcp-fb:45 transport-cc
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 nack
a=rtcp-fb:45 nack pli
a=fmtp:45 level-idx=5;profile=0;tier=0
a=rtpmap:46 rtx/90000
a=fmtp:46 apt=45
a=rtpmap:47 AV1/90000
a=rtcp-fb:47 goog-remb
a=rtcp-fb:47 transport-cc
a=rtcp-fb:47 ccm fir
a=rtcp-fb:47 nack
a=rtcp-fb:47 nack pli
a=fmtp:47 level-idx=5;profile=1;tier=0
a=rtpmap:48 rtx/90000
a=fmtp:48 apt=47
a=rtpmap:119 H264/90000
a=rtcp-fb:119 goog-remb
a=rtcp-fb:119 transport-cc
a=rtcp-fb:119 ccm fir
a=rtcp-fb:119 nack
a=rtcp-fb:119 nack pli
a=fmtp:119 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=119
a=rtpmap:121 H264/90000
a=rtcp-fb:121 goog-remb
a=rtcp-fb:121 transport-cc
a=rtcp-fb:121 ccm fir
a=rtcp-fb:121 nack
a=rtcp-fb:121 nack pli
a=fmtp:121 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=64001f
a=rtpmap:122 rtx/90000
a=fmtp:122 apt=121
a=rtpmap:49 H265/90000
a=rtcp-fb:49 goog-remb
a=rtcp-fb:49 transport-cc
a=rtcp-fb:49 ccm fir
a=rtcp-fb:49 nack
a=rtcp-fb:49 nack pli
a=fmtp:49 level-id=180;profile-id=1;tier-flag=0;tx-mode=SRST
a=rtpmap:50 rtx/90000
a=fmtp:50 apt=49
a=rtpmap:51 H265/90000
a=rtcp-fb:51 goog-remb
a=rtcp-fb:51 transport-cc
a=rtcp-fb:51 ccm fir
a=rtcp-fb:51 nack
a=rtcp-fb:51 nack pli
a=fmtp:51 level-id=180;profile-id=2;tier-flag=0;tx-mode=SRST
a=rtpmap:52 rtx/90000
a=fmtp:52 apt=51
a=rtpmap:123 red/90000
a=rtpmap:124 rtx/90000
a=fmtp:124 apt=123
a=rtpmap:125 ulpfec/90000
a=rtpmap:53 flexfec-03/90000
a=rtcp-fb:53 goog-remb
a=rtcp-fb:53 transport-cc
a=fmtp:53 repair-window=10000000

The SDP answer (after enabeling 2-way audio) on chrome:

WebRTC: answer v=0
o=- 4885768737105245046 1751915639 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic:WMS*
a=fingerprint:sha-256 46:44:E4:47:07:D6:D6:B8:CA:A5:7A:CE:56:E9:F3:CC:27:BF:E3:89:77:BB:F7:1A:E2:F4:58:76:FC:8B:B0:B9
a=extmap-allow-mixed
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVPF 0
c=IN IP4 0.0.0.0
a=setup:active
a=mid:0
a=ice-ufrag:JYBCaWWxVyYduhJh
a=ice-pwd:QcPTQjoldUwrGIVgBMBjoOrgraGMDRZo
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:0 PCMU/8000
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=ssrc:3451033683 cname:go2rtc
a=ssrc:3451033683 msid:go2rtc go2rtc-audio
a=ssrc:3451033683 mslabel:go2rtc
a=ssrc:3451033683 label:go2rtc-audio
a=msid:go2rtc go2rtc-audio
a=sendrecv
m=video 9 UDP/TLS/RTP/SAVPF 103 109 119 49 51
c=IN IP4 0.0.0.0
a=setup:active
a=mid:1
a=ice-ufrag:JYBCaWWxVyYduhJh
a=ice-pwd:QcPTQjoldUwrGIVgBMBjoOrgraGMDRZo
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:103 H264/90000
a=fmtp:103 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtcp-fb:103 goog-remb 
a=rtcp-fb:103 ccm fir
a=rtcp-fb:103 nack 
a=rtcp-fb:103 nack pli
a=rtcp-fb:103 nack 
a=rtcp-fb:103 nack pli
a=rtcp-fb:103 transport-cc 
a=rtpmap:109 H264/90000
a=fmtp:109 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtcp-fb:109 goog-remb 
a=rtcp-fb:109 ccm fir
a=rtcp-fb:109 nack 
a=rtcp-fb:109 nack pli
a=rtcp-fb:109 nack 
a=rtcp-fb:109 nack pli
a=rtcp-fb:109 transport-cc 
a=rtpmap:119 H264/90000
a=fmtp:119 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f
a=rtcp-fb:119 goog-remb 
a=rtcp-fb:119 ccm fir
a=rtcp-fb:119 nack 
a=rtcp-fb:119 nack pli
a=rtcp-fb:119 nack 
a=rtcp-fb:119 nack pli
a=rtcp-fb:119 transport-cc 
a=rtpmap:49 H265/90000
a=fmtp:49 level-id=180;profile-id=1;tier-flag=0;tx-mode=SRST
a=rtcp-fb:49 goog-remb 
a=rtcp-fb:49 ccm fir
a=rtcp-fb:49 nack 
a=rtcp-fb:49 nack pli
a=rtcp-fb:49 nack 
a=rtcp-fb:49 nack pli
a=rtcp-fb:49 transport-cc 
a=rtpmap:51 H265/90000
a=fmtp:51 level-id=180;profile-id=2;tier-flag=0;tx-mode=SRST
a=rtcp-fb:51 goog-remb 
a=rtcp-fb:51 ccm fir
a=rtcp-fb:51 nack 
a=rtcp-fb:51 nack pli
a=rtcp-fb:51 nack 
a=rtcp-fb:51 nack pli
a=rtcp-fb:51 transport-cc 
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=ssrc:2846699528 cname:go2rtc
a=ssrc:2846699528 msid:go2rtc go2rtc-video
a=ssrc:2846699528 mslabel:go2rtc
a=ssrc:2846699528 label:go2rtc-video
a=msid:go2rtc go2rtc-video
a=sendonly

The SDP offer (after enabeling 2-way audio) on firefox:

WebRTC: start webRtcOffer v=0
o=mozilla...THIS_IS_SDPARTA-99.0 449997163758058029 1 IN IP4 0.0.0.0
s=-
t=0 0
a=fingerprint:sha-256 D6:95:14:0B:28:62:B8:24:06:EA:80:BA:4A:E3:63:6D:FA:67:D8:35:29:FC:97:0B:92:AD:BF:CF:07:EC:DE:C9
a=group:BUNDLE 0 1
a=ice-options:trickle
a=msid-semantic:WMS *
m=audio 9 UDP/TLS/RTP/SAVPF 109 0 8 9 101
c=IN IP4 0.0.0.0
a=sendrecv
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2/recvonly urn:ietf:params:rtp-hdrext:csrc-audio-level
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap-allow-mixed
a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=fmtp:101 0-15
a=ice-pwd:48480b42e986de3d96464eeb3c3154cc
a=ice-ufrag:4fac9cbd
a=mid:0
a=msid:- {91c3c42c-f297-4087-8d84-95f843745d08}
a=rtcp-mux
a=rtpmap:109 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:9 G722/8000/1
a=rtpmap:101 telephone-event/8000/1
a=setup:actpass
a=ssrc:626550916 cname:{34b09abe-c302-4064-9381-8b8eb6725a58}
m=video 9 UDP/TLS/RTP/SAVPF 126 127 105 106 120 124 121 125 97 98 103 104 99 100 123 122 119
c=IN IP4 0.0.0.0
a=recvonly
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:5 urn:ietf:params:rtp-hdrext:toffset
a=extmap:6/recvonly http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:7 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap-allow-mixed
a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=fmtp:105 profile-level-id=42001f;level-asymmetry-allowed=1;packetization-mode=1
a=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1
a=fmtp:103 profile-level-id=42001f;level-asymmetry-allowed=1
a=fmtp:127 apt=126
a=fmtp:106 apt=105
a=fmtp:120 max-fs=12288;max-fr=60
a=fmtp:124 apt=120
a=fmtp:121 max-fs=12288;max-fr=60
a=fmtp:125 apt=121
a=fmtp:98 apt=97
a=fmtp:104 apt=103
a=fmtp:100 apt=99
a=fmtp:119 apt=122
a=ice-pwd:48480b42e986de3d96464eeb3c3154cc
a=ice-ufrag:4fac9cbd
a=mid:1
a=rtcp-fb:126 nack
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 ccm fir
a=rtcp-fb:126 goog-remb
a=rtcp-fb:126 transport-cc
a=rtcp-fb:105 nack
a=rtcp-fb:105 nack pli
a=rtcp-fb:105 ccm fir
a=rtcp-fb:105 goog-remb
a=rtcp-fb:105 transport-cc
a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir
a=rtcp-fb:120 goog-remb
a=rtcp-fb:120 transport-cc
a=rtcp-fb:121 nack
a=rtcp-fb:121 nack pli
a=rtcp-fb:121 ccm fir
a=rtcp-fb:121 goog-remb
a=rtcp-fb:121 transport-cc
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=rtcp-fb:97 ccm fir
a=rtcp-fb:97 goog-remb
a=rtcp-fb:97 transport-cc
a=rtcp-fb:103 nack
a=rtcp-fb:103 nack pli
a=rtcp-fb:103 ccm fir
a=rtcp-fb:103 goog-remb
a=rtcp-fb:103 transport-cc
a=rtcp-fb:99 nack
a=rtcp-fb:99 nack pli
a=rtcp-fb:99 ccm fir
a=rtcp-fb:99 goog-remb
a=rtcp-fb:99 transport-cc
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=rtcp-fb:123 ccm fir
a=rtcp-fb:123 goog-remb
a=rtcp-fb:123 transport-cc
a=rtcp-fb:122 nack
a=rtcp-fb:122 nack pli
a=rtcp-fb:122 ccm fir
a=rtcp-fb:122 goog-remb
a=rtcp-fb:122 transport-cc
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:126 H264/90000
a=rtpmap:127 rtx/90000
a=rtpmap:105 H264/90000
a=rtpmap:106 rtx/90000
a=rtpmap:120 VP8/90000
a=rtpmap:124 rtx/90000
a=rtpmap:121 VP9/90000
a=rtpmap:125 rtx/90000
a=rtpmap:97 H264/90000
a=rtpmap:98 rtx/90000
a=rtpmap:103 H264/90000
a=rtpmap:104 rtx/90000
a=rtpmap:99 AV1/90000
a=rtpmap:100 rtx/90000
a=rtpmap:123 ulpfec/90000
a=rtpmap:122 red/90000
a=rtpmap:119 rtx/90000
a=setup:actpass
a=ssrc:654813769 cname:{34b09abe-c302-4064-9381-8b8eb6725a58}

The SDP answer (after enabeling 2-way audio) on firefox:

WebRTC: answer v=0
o=- 6524466964061306624 1751915792 IN IP4 0.0.0.0
s=-
t=0 0
a=msid-semantic:WMS*
a=fingerprint:sha-256 34:2E:20:94:D7:8E:0C:81:8E:14:0D:FD:16:55:DC:93:4D:59:99:B6:E4:FC:DB:7B:89:A9:F1:49:09:5A:F9:27
a=group:BUNDLE 0 1
m=audio 9 UDP/TLS/RTP/SAVPF 0
c=IN IP4 0.0.0.0
a=setup:active
a=mid:0
a=ice-ufrag:KVPYmUBRizVNZQni
a=ice-pwd:IizFCAsknXMeXyLwNlLVlbYFUJOXjFux
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:0 PCMU/8000
a=ssrc:1211358887 cname:go2rtc
a=ssrc:1211358887 msid:go2rtc go2rtc-audio
a=ssrc:1211358887 mslabel:go2rtc
a=ssrc:1211358887 label:go2rtc-audio
a=msid:go2rtc go2rtc-audio
a=sendrecv
m=video 9 UDP/TLS/RTP/SAVPF 126 105
c=IN IP4 0.0.0.0
a=setup:active
a=mid:1
a=ice-ufrag:KVPYmUBRizVNZQni
a=ice-pwd:IizFCAsknXMeXyLwNlLVlbYFUJOXjFux
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:126 H264/90000
a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=rtcp-fb:126 goog-remb 
a=rtcp-fb:126 ccm fir
a=rtcp-fb:126 nack 
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 nack 
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 transport-cc 
a=rtpmap:105 H264/90000
a=fmtp:105 profile-level-id=42001f;level-asymmetry-allowed=1;packetization-mode=1
a=rtcp-fb:105 goog-remb 
a=rtcp-fb:105 ccm fir
a=rtcp-fb:105 nack 
a=rtcp-fb:105 nack pli
a=rtcp-fb:105 nack 
a=rtcp-fb:105 nack pli
a=rtcp-fb:105 transport-cc 
a=extmap:7 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=ssrc:1991047866 cname:go2rtc
a=ssrc:1991047866 msid:go2rtc go2rtc-video
a=ssrc:1991047866 mslabel:go2rtc
a=ssrc:1991047866 label:go2rtc-video
a=msid:go2rtc go2rtc-video
a=sendonly

[starkillerOG]

@AlexxIT any help would be greathly appreciated!

[AlexxIT]

As I know, you can't increase any information in SDP exchange. You can only decrease it.
For example you can change sendrecv to sendonly or recvonly. Or disable track. Or leave only few codecs.
But you can't add new tracks, new codecs or new direction.

[starkillerOG]

It does work for Chrome do.
Besides as far as I read about webrtc on google, you are supposed to be able to add a track.
Then the onnegotiationneeded will be fired and the new SDP will be exhanged.

[starkillerOG]

@AlexxIT could it be that go2rtc is ignoring the iceRestart: true when receiving the new offer (createOffer)?

If I look at the console, during the first negotiation I get the following order of events:

1. createOffer
2. setLocalDescription
3. signal offer
4. ice gathering
5. local ice candidates
6. receive answer
7. setRemoteDescription
8. remote ice candidate
9. ice connection state change checking
10. ice connection state change connected
11. ice gathering state change complete

However when doing the renegotioation I get the following order of events:

1. createOffer
2. setLocalDescription
3. ice connection state change checking
4. signal offer
5. ice gathering
6. local ice candidates
7. receive answer
8. setRemoteDescription
9. ice connection state change connected
10. remote ice candidate
11. more local ice candidates
12. ice gat hering state change complete

What is surprising to me is that the ice connection state change checking happens way to early I think, moreover the ice connection state change connected happens before receiving the first remote ice candidates. Therefore I think something (like the ice candidates) is not properly cleared and it is connecting to one of the old candidates......

[AlexxIT]

According to this issue, important changes have been made in pion/webrtc v4.1.3.
pion/webrtc#1675

Can you try master version? I have updated all the dependencies
https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#go2rtc-dev-version

If this doesn't help, then let's try something with iceRestart.

[starkillerOG]

@AlexxIT I tried the master version and the following error is now indeed gone:
2025-07-07 18:19:27.671 WARNING (MainThread) [homeassistant.components.go2rtc.server] pc ERROR: 2025/07/07 18:19:27 Incoming unhandled RTP ssrc(1894855586), OnTrack will not be fired. mid RTP Extensions required for Simulcast

So we are one step closer :)

Unfortunately the stream still stops when using firefox.....
But I am not getting any errors anymore.

[starkillerOG]

On chrome it continues to work as expected with the master version.

[starkillerOG]

Chrome with master version after enabeling the microphone and renegotiation:

Chrome with current go2rtc version HA is using after enabeling the microphone and renegotiation:

As you can see, on the old version it opens up a second webrtc stream (it does work)
While on the new master version it seems to be cleaner and just adds the backchannel to the existing webrtc

[starkillerOG]

On firefox with the new master go2rtc version I get the following:
Before pressing the microphone button: video and audio working

Emediatly after pressing the microphone button: The stream freezes (static video, no audio, no backchannel audio)

After a few seconds when the microphone button is pressed: The stream freezes (static video, no audio, no backchannel audio)

[AlexxIT]

Firefox's WebRTC support is far from ideal.

It's hard for me to say anything here, I've never encountered the renegotiation process in WebRTC and didn't even know it was supported.

Do you have the opportunity to test Safari? I wonder what will happen there.

[starkillerOG]

Unfortunetly I don't have any apple devices, so can't run Safari.
I could try Edge or Opera if you are interested?

[AlexxIT]

I think in terms of browser popularity, Safari is much higher than Firefox. We can try to do something with iceRestart.

[starkillerOG]

I am learning as I go, and currently I think go2rtc is interpetting the new offer from the renegotiation as a seperate webrtc stream instead of "updating" the already existing webrtc stream.

See above pictures.

Do you know how go2rtc decides if it will make a new webrtc or update the existing one?

[AlexxIT]

If I understand your PR correctly, go2rtc creates a new client. Perhaps it closes the old client very quickly.
You can check stream JSON info. Every client has unique ID.