maxLiveSyncPlaybackRate does not seem to enable "catch up"

[jeffsf]

What do you want to do with Hls.js?

When streaming from an IP cam, there are times that a user would want to go back and review a very recent segment, then return to "live" monitoring. Reading the documentation, I interpreted maxLiveSyncPlaybackRate in the range of 1-2 to imply that the playback speed would be automatically increased if the estimated latency exceeded the target latency. I am not seeing any change in my local code, or when using https://hls-js.netlify.app/demo/ (options shown below)

Edit: I will probably change this into an enhancement request, as it initially appears from a quick read of the code that the catch-up adjustment is only made under very limited conditions. It may be worth parameterizing both for a couple use cases as well as making it settable during run time. ref: https://github.com/video-dev/hls.js/blob/master/src/controller/latency-controller.ts#L214

{
  "debug": true,
  "enableWorker": true,
  "lowLatencyMode": true,
  "backBufferLength": 90,
  "liveSyncDurationCount": 3,
  "maxLiveSyncPlaybackRate": 1.99
}

1.99 was checked after not seeing the expected behavior at 2

HLS content is generated with rtsp-simple-server with an Axis M1011-W generating the source H.264-compressed stream.

I have tried adjusting liveMaxLatencyDurationCount, but its behavior seems consistent with its documentation, where if the latency gets too long, it "snaps to" nearly the front of the stream.

Prior to "clicking back", I see

• Max Latency: Infinity
• Target Latency: 7.000
• Latency: 2-3 seconds
• Drift: close to 1.0
• Playback rate: 1.00
• Program Date Time a bit over 3 seconds ago

After clicking back

• Max Latency: Infinity
• Target Latency: 7.000
• Latency increases to around 32 seconds (where I clicked)
• Drift remains close to 1.0
• Playback rate: 1.00 (unchanged)
• Program Date Time near 32 seconds ago

macOS 12.3.1 (Monterey)
Chrome 100.0.4896.127

What have you tried so far?

See above for exploration with "official demo" page.

###############################################
# HLS parameters

# Disable support for the HLS protocol.
hlsDisable: no
# Address of the HLS listener.
hlsAddress: :8888
# By default, HLS is generated only when requested by a user.
# This option allows to generate it always, avoiding the delay between request and generation.
hlsAlwaysRemux: yes
# Number of HLS segments to generate.
# Increasing segments allows more buffering,
# Decreasing segments decreases latency.
hlsSegmentCount: 100
# Minimum duration of each segment.
# The final segment duration is also influenced by the interval between IDR frames,
# since the server changes the segment duration to include at least a IDR frame in each one.
hlsSegmentDuration: 1s
# Maximum size of each segment.
# This prevents RAM exhaustion.
hlsSegmentMaxSize: 50M
# Value of the Access-Control-Allow-Origin header provided in every HTTP response.
# This allows to play the HLS stream from an external website.
hlsAllowOrigin: '*'

Earlier, local configuration (originally from rtsp-simple-server†) explored several variants, the last of which was

        const hls = new Hls({
            progressive: true,
            liveSyncDurationCount: 3,
            // liveMaxLatencyDurationCount: 4,                                                                         
            maxLiveSyncPlaybackRate: 2,
            liveDurationInfity: true,
        });

† Ref: https://github.com/aler9/rtsp-simple-server/blob/main/internal/core/hls_muxer.go#L30

[jeffsf]

I'm possibly misinterpreting things, but it isn't clear to me that

// Playback further than one target duration from target can be considered DVR playback.

is reasonably possible at any playback rate other than 1.

In src/controller/latency-controller.ts

private timeupdate() will be skipped if wither not in lowLatencyMode or if the "catch-up" feature is disabled with maxLiveSyncPlaybackRate = 1

later

    if (
      levelDetails.live &&
      inLiveRange &&
      distanceFromTarget > 0.05 &&
      this.forwardBufferLength > 1
    ) {
      const max = Math.min(2, Math.max(1.0, maxLiveSyncPlaybackRate));
      const rate =
        Math.round(
          (2 / (1 + Math.exp(-0.75 * distanceFromTarget - this.edgeStalled))) *
            20
        ) / 20;
      media.playbackRate = Math.min(max, Math.max(1, rate));
    } else if (media.playbackRate !== 1 && media.playbackRate !== 0) {
      media.playbackRate = 1;
    }

Here, if outside of the pull-in range of roughly 2 * targetLatency from the leading edge of the stream, if the playback rate is not one or zero, it is set to one.

This implies that if the user is "in the history" of a live stream, they can only pause the stream or play at normal rate, not, for example, double rate to scan through it, as long as lowLatencyMode is set.

If this were the only place that lowLatencyMode was used, it could be managed and cleared whenever "too far" back due to control requesting a move into the past, a pause of the stream, and probably a couple more use cases.

However, it is also used in base-playlist-controller.ts and base-stream-controller.ts so it may have unintended side effects to clear lowLatencyMode for the duration of the user's examination of the back history of a live stream.

(The comment reference in abr-controller.ts seems detached .)

Am I missing a more obvious bettter way to:

• Have the playback of the low-latency, live stream "catch up" automatically from relatively small delays
• Allow the user to examine the back history of the stream, within reasonable bounds ("a minute or so", limited by cached segments and liveMaxLatencyDurationCount or liveMaxLatencyDuration)

---

Edit: I could also actively manage maxLiveSyncPlaybackRate, but that doesn't feel much better.

[robwalch]

Hi @jeffsf,

I'm possibly misinterpreting things, but it isn't clear to me that // Playback further than one target duration from target can be considered DVR playback.

I may have initially misread or summarized this issue as wanting the configured rate to be applied no matter how far back from the edge the playhead is.

Here, if outside of the pull-in range of roughly 2 * targetLatency from the leading edge of the stream, if the playback rate is not one or zero, it is set to one.

} else [...] {
 media.playbackRate = 1;
}

Looking at this more carefully, I think the issue is more that the rate is forced back to 1 when playback is further away. If hls.js only reset the rate once on seek, allowing you (or the user) to set their own rate (until arriving near the edge again) would that solve the issue? If so we could file or convert this to a bug - something like "Do not force a rate of 1 when playback is far from the live edge with maxLiveSyncPlaybackRate".