Reolink Duo 2 - Event playback is slow

[baudneo]

I don't have web dev experience, so I am not equipped to debug this myself. I am running master branch from latest commit: c04b4b9

All other monitors and events work as expected. Just installed a reolink duo 2v, the one with dual lenses that exposes a single rtsp stream with both streams stitched together to create a 180 degree wide view. I am running CBR 9216kbps, constant 20 fps, 5120x1552 resolution on the main stream.

The events are stored on a dedicated SAS3 SSD with an iostat util% of 7 infrequently, so IO doesn't seem to be an issue. Host server has 10G link on LAN, so that also rules out a bandwidth issue (I host via WAN 1Gbit, but have the same behaviour on LAN).

Streaming directly from the camera is fine and instant, it is just video playback. Tested using chrome and FF. Videos are pass through with H264. VLC can play the downloaded event video without problem.

In dev tools > network, I can see it repeatedly requesting xhr chunks of 493 B ->

The playback fps bounces between 0.xx - 7 ish fps.

Not sure what I should be looking at to help debug/fix further, any input is appreciated.

Edit: forgot to mention it is H265 HEVC codec

ffprobe output

Input #0, rtsp, from 'rtsp://10.0.1.14:8554/duo2v':
  Metadata:
    title           : go2rtc/1.9.2
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: hevc (Main), yuv420p(tv), 5120x1552, 30 tbr, 90k tbn, 90k tbc
  Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp

[baudneo]

When I set the resolution lower, the codec is set to h264 and events playback without issue in the ZM event viewer. So, I guess its something to do with H265 event playback?

new ffprobe output:

Input #0, rtsp, from 'rtsp://10.0.1.14:8554/duo2v':
  Metadata:
    title           : go2rtc/1.9.2
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 4096x1248, 90k tbr, 90k tbn, 180k tbc
  Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp

[connortechnology]

h265 playback happens through zms. The code to do this is not great.

I think the main issue is that we have timestamp data in the db from the days when we store jpegs. So we are constantly saying, give me the frame for this timestamp and my code in zm_ffmpeg_input goes seeking around in the mp4 trying to efficiently find the keyframe before that time and then decode until we get it.

[connortechnology]

There is a dev version of chrome that will natively play back h265. However I really do need to improve zms playback.

[baudneo]

I thought newer chrome releases generally had h265 playback, I think linux can only use HW decoding (no SW decoding at all)? As it turns out, its an oversight on my part. I assumed newer AMD iGPU's had HEVC HW decoding, but apparently not mine my configuration (look into AMDGPU in arch linux docs). Worst case, I install an Nvidia gpu for h265.

The weird thing is, how did the stream load at all? When I try and play test HEVC videos from test sites, they dont even load, just the audio track comes through. Is zmc/zms using ffmpeg to do some magic?

Android has HEVC HW decoding support in chrome, when testing, it has the same playback issue as desktop for events. So, it probably is as you described for that.

For now, the lower res h264 stream is fine for me. However, the new reolink Duo 3 (16MP) main stream is H265 only, so anyone using one of those pixel vacuums will have issues with event playback in frontend, just FYI.

Unrelated topic, really glad to see MQTT working, opens up all sorts of avenues for things to integrate.

[connortechnology]

Yeah if its h265 we don't use the video tag, we use zms to generate an mjpeg stream. Which is not great.

On my 12th gen intel laptop, using a dev snapshot of chrome, with some command line magic, it plays h265 natively. I think under windows and mac the situation is better, but still not generally available.

My systems here are all av1 now which is nice....

Yeah the mqtt has been around time, but really needs fleshing out.

[baudneo]

I'm back working on zomi in my freetime, I have a bunch of ideas. Currently working on a mqtt-daemon alongside the CLI exec. Next is easier install, default config, docs, endpoint user management and then work on performance/extra backends. Adding other data transports between server and client during performance work, HTTP is now the bottleneck there. I'm thinking SHM if server and client on same host, MQTT/gRPC if LAN/WAN with the HTTP REST API for testing/fallback.

Current mqtt work in mqtt-daemon branch.

  03/02/25 21:11:35.686 zomi:C DEBUG[main:2577]> mqtt:start: Starting MQTT client...
Could not gracefully disconnect: 4. Forcing disconnection.
  03/02/25 21:11:35.687 zomi:C DEBUG[main:2512]> mqtt:connect: Connecting to MQTT broker...
  03/02/25 21:11:35.690 zomi:C INFO[main:625]> Client:event: Starting, waiting for ZM motion events...
  03/02/25 21:11:35.692 zomi:C INFO[main:2530]> mqtt:connect: Connected to MQTT broker: 10.0.1.20 port: 1883
  03/02/25 21:11:35.692 zomi:C INFO[main:2592]> mqtt:subscribe: Subscribing to MQTT topics: ['ZoneMinder/monitor/#'] ...
  03/02/25 21:11:35.694 zomi:C INFO[main:2598]> mqtt:rcv: Waiting for MQTT messages...
  03/02/25 21:11:41.352 zomi:C DEBUG[main:2603]> mqtt:rcv: [QoS: 0 | Topic: ZoneMinder/monitor/5] Payload => b'event start: 538880'
  03/02/25 21:11:41.352 zomi:C INFO[main:628]> Client:event: ZM event (start) received: ID: 538880 from Monitor: 5
  03/02/25 21:11:41.353 zomi:C INFO[main:630]> Client:event: this is where the detect code goes
  03/02/25 21:11:49.734 zomi:C DEBUG[main:2603]> mqtt:rcv: [QoS: 0 | Topic: ZoneMinder/monitor/2] Payload => b'Capturing at 14.28 fps, capturing bandwidth 395433bytes/sec Analysing at 2.86 fps'
  03/02/25 21:11:52.314 zomi:C DEBUG[main:2603]> mqtt:rcv: [QoS: 0 | Topic: ZoneMinder/monitor/5] Payload => b'event end: 538880'
  03/02/25 21:11:52.314 zomi:C INFO[main:628]> Client:event: ZM event (end) received: ID: 538880 from Monitor: 5
  03/02/25 21:11:52.314 zomi:C DEBUG[main:633]> Client:event: ZM event end, no action taken
  03/02/25 21:12:07.639 zomi:C DEBUG[main:2603]> mqtt:rcv: [QoS: 0 | Topic: ZoneMinder/monitor/6] Payload => b'Capturing at 11.95 fps, capturing bandwidth 350377bytes/sec Analysing at 2.99 fps'

FYI: YOLO11 works with zomi as a drop in to the yolov8 onnx and tensor-rt config. Export yolo11 as .onnx from their docs and then load the model into ort config. tensorrt also works with yolov8 output_type, if you have that all set-up.

  - name: yolov11x ort
    description: "Ultralytics YOLO v11 Xtra Large pretrained ONNX model on onnxruntime"
    enabled: yes
    # - Use onnxruntime backend: ort
    framework: ort

    type_of: object

    processor: gpu
    #    gpu_idx: 0
    #- Possible AMD support via ROCm
    #    gpu_framework: cuda

    input: "${MODEL_DIR}/yolo/yolo11x.onnx"
    #classes: path/to/classes.file

    # ** NOTE: MAKE SURE to set the proper model input size or it will throw errors!
    height: 640
    width: 640

    detection_options:
      confidence: 0.33
      nms:
        enabled: yes
        threshold: 0.44

# Example Tensor-RT config
  - name: yolo11x trt
    enabled: yes
    description: "TensorRT optimized YOLO v11 Xtra Large pretrained model"
    input: "${MODEL_DIR}/yolo/yolo11x.trt"
    #    gpu_idx: 0
    framework: trt
    sub_framework:
    processor: gpu
    type_of: object
    height: 640
    width: 640
    output_type: yolov8

    detection_options:
      confidence: 0.33
      nms:
        enabled: yes
        threshold: 0.44

[connortechnology]

There is some interesting zomi/AI related stuff that I'd like to discuss with you private-ishly.

[baudneo]

Ok, I am available tmrw after 4pm mountain time and most of the weekend. Just got out of technical training after 5 months

[connortechnology]

Apparently firefox 136 just dropped and includes hwaccel for linux on amd gpus... might be interesting to try out.

[connortechnology]

I'm vacationing in Florida... schedule is all over the place. Can chat whenever we are both aroudn or just via DM etc. Pick your platform. github is not great for DM.

[baudneo]

What is the best platform to get ahold of you on? I don't have any messengers installed. WhatsApp? Telegram? Discord? Slack? Teams?

[connortechnology]

I'm still on all places I used to be. Slack ,discord. I have teams, and whatsapp. Basic email is a option.

P