Missing audio in history playback

[tpjanssen]

I was running an old 0.13 installation, where camera streams were restreamed to my 0.15.0 beta installation. I'm moving away from this, now my 0.15.0 beta connects directly to my cameras. Having some problems with the audio of one of cameras now. The live view of the camera gives audio, however playing back history doesn't.

Relevant camera config:

go2rtc:
  streams:
    voordeur:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
    voordeur_detect:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0

cameras:
  voordeur:
    webui_url: http://192.168.5.143/
    mqtt:
      quality: 95
    ui:
      order: 1
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/voordeur
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/voordeur_detect
          input_args: preset-rtsp-restream
          roles:
            - detect
    live:
      stream_name: voordeur

The go2rtc API gives this on first "voordeur" stream:

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.5.143:554/cam/realmonitor?channel=1\u0026subtype=1/",
      "remote_addr": "192.168.5.143:554",
      "user_agent": "go2rtc/1.9.2",
      "sdp": "v=0\r\no=- 2261922225 2261922225 IN IP4 0.0.0.0\r\ns=Media Server\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\na=control:*\r\na=packetization-supported:DH\r\na=rtppayload-supported:DH\r\na=range:npt=now-\r\na=x-packetization-supported:IV\r\na=x-rtppayload-supported:IV\r\nm=video 0 RTP/AVP 96\r\na=control:trackID=0\r\na=framerate:25.000000\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=420028;sprop-parameter-sets=Z0IAKOkA8ARPywgAAB9IAAYcECAA,aMqPIAA=\r\na=recvonly\r\nm=audio 0 RTP/AVP 0\r\na=control:trackID=1\r\na=rtpmap:0 PCMU/8000\r\na=recvonly\r\nm=audio 0 RTP/AVP 0\r\na=control:trackID=5\r\na=rtpmap:0 PCMU/8000\r\na=sendonly\r\n",
      "medias": [
        "video, recvonly, H.264 Baseline 4.0",
        "audio, recvonly, PCMU/8000",
        "audio, sendonly, PCMU/8000"
      ],
      "receivers": [
        "96 H264, bytes=101795130, senders=1"
      ],
      "recv": 102650682
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://127.0.0.1:8554/voordeur",
      "remote_addr": "127.0.0.1:41758",
      "user_agent": "FFmpeg Frigate/0.15.0-87e7b62",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.2\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=420028;sprop-parameter-sets=Z0IAKOkA8ARPywgAAB9IAAYcECAA,aMqPIAA=\r\na=control:trackID=0\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=101795130, receivers=1"
      ],
      "send": 102934586
    }
  ]
}

And the other "voordeur_detect" stream:

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.5.143:554/cam/realmonitor?channel=1\u0026subtype=0/",
      "remote_addr": "192.168.5.143:554",
      "user_agent": "go2rtc/1.9.2",
      "sdp": "v=0\r\no=- 2261922225 2261922225 IN IP4 0.0.0.0\r\ns=Media Server\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\na=control:*\r\na=packetization-supported:DH\r\na=rtppayload-supported:DH\r\na=range:npt=now-\r\na=x-packetization-supported:IV\r\na=x-rtppayload-supported:IV\r\nm=video 0 RTP/AVP 96\r\na=control:trackID=0\r\na=framerate:10.000000\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=42001F;sprop-parameter-sets=Z0IAH+kAoAt0IAAAfSAACcaAgAA=,aMqPIAA=\r\na=recvonly\r\nm=audio 0 RTP/AVP 0\r\na=control:trackID=1\r\na=rtpmap:0 PCMU/8000\r\na=recvonly\r\n",
      "medias": [
        "video, recvonly, H.264 Baseline 3.1",
        "audio, recvonly, PCMU/8000"
      ],
      "receivers": [
        "96 H264, bytes=34672672, senders=1"
      ],
      "recv": 34978804
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://127.0.0.1:8554/voordeur_detect",
      "remote_addr": "127.0.0.1:41806",
      "user_agent": "FFmpeg Frigate/0.15.0-87e7b62",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.2\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=42001F;sprop-parameter-sets=Z0IAH+kAoAt0IAAAfSAACcaAgAA=,aMqPIAA=\r\na=control:trackID=0\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=34672672, receivers=1"
      ],
      "send": 35077328
    }
  ]
}

Why I'm a missing audio in the history playback? I would expect it to be the same stream?

I'm using "Chrome 131.0.6778.205 (Official Build) (64-bit)", but other browsers have the same outcome.

[NickM-27]

You're missing audio because the camera provides pcma audio, you need to add a second aac transcode

[NickM-27]

Nothing has changed, aac has always been required for recording audio

[tpjanssen]

But based on the docs I would expect preset preset-record-generic-audio-copy to copy the pcmu stream?

[NickM-27]

No, because as I said, recordings REQUIRE aac. MP4 is not compatible with pcmu so it is not usable for recordings

[NickM-27]

One other issue I just realized is that your streams are reversed. Actually, subtype=0 is the record stream and subtype=1 is the sub stream

[tpjanssen]

No, because as I said, recordings REQUIRE aac. MP4 is not compatible with pcmu so it is not usable for recordings

OK. Now it makes sense to me why I need AAC.

One other issue I just realized is that your streams are reversed. Actually, subtype=0 is the record stream and subtype=1 is the sub stream

Not on my Dahua VTO. On the second stream I can have a higher resolution than on the main stream.

That having said. Now I see this in my go2rtc API:

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.5.143:554/cam/realmonitor?channel=1\u0026subtype=1/",
      "remote_addr": "192.168.5.143:554",
      "user_agent": "go2rtc/1.9.2",
      "sdp": "v=0\r\no=- 2251946648 2251946648 IN IP4 0.0.0.0\r\ns=Media Server\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\na=control:*\r\na=packetization-supported:DH\r\na=rtppayload-supported:DH\r\na=range:npt=now-\r\na=x-packetization-supported:IV\r\na=x-rtppayload-supported:IV\r\nm=video 0 RTP/AVP 96\r\na=control:trackID=0\r\na=framerate:25.000000\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=420028;sprop-parameter-sets=Z0IAKOkA8ARPywgAAB9IAAYcECAA,aMqPIAA=\r\na=recvonly\r\nm=audio 0 RTP/AVP 97\r\na=control:trackID=1\r\na=rtpmap:97 L16/16000\r\na=recvonly\r\nm=audio 0 RTP/AVP 97\r\na=control:trackID=5\r\na=rtpmap:97 L16/16000\r\na=sendonly\r\n",
      "medias": [
        "video, recvonly, H.264 Baseline 4.0",
        "audio, recvonly, L16/16000",
        "audio, sendonly, L16/16000"
      ],
      "receivers": [
        "96 H264, bytes=8369929112, senders=1",
        "97 L16/16000, bytes=263631360, senders=0"
      ],
      "senders": [
        "97 L16/16000, bytes=0, receivers=0"
      ],
      "recv": 8708971668
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://127.0.0.1:8554/voordeur",
      "remote_addr": "127.0.0.1:58512",
      "user_agent": "FFmpeg Frigate/0.15.0-87e7b62",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.2\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=420028;sprop-parameter-sets=Z0IAKOkA8ARPywgAAB9IAAYcECAA,aMqPIAA=\r\na=control:trackID=0\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=8369929112, receivers=1"
      ],
      "send": 8463675320
    }
  ]
}

So the codec also seems to switch for whatever reason....

[hawkeye217]

See the docs: https://docs.frigate.video/troubleshooting/faqs/#audio-in-recordings

[Silverstream89]

I have the same Problem:

i have 2 Reolink Cameras at Frigate and TrueNas

With this code:

mqtt:
enabled: true

record:
enabled: true
retain:
days: 180
mode: all
events:
retain:
default: 30
mode: all

cameras:
Cam1: # <-----Name of cam1
enabled: true
ffmpeg:
inputs:
- path: rtsp://Acc:[email protected]:554/Preview_01_main
roles:
- detect
detect:
enabled: true
width: 2304
height: 1296
fps: 20

Cam2: # <------ Name 2nd Cam
enabled: true
ffmpeg:
inputs:
- path: rtsp://Acc:[email protected]:554/Preview_01_main
roles:
- detect
detect:
enabled: true
width: 2304
height: 1296
fps: 20

How can i Add Audio to Stream an Record?

Thanks so far

Greetings

[hawkeye217]

See the documentation: https://docs.frigate.video/troubleshooting/faqs/#audio-in-recordings

[Silverstream89]

Thanks for the Reply.

Sadly im not good at coding... Where do i have to put those lines in?

[Silverstream89]

[NickM-27]

put them at the top or before cameras