TTL and DCSM reloading

[bbert]

I have some considerations on content steering specification about TTL.

1. First, specification is a bit confusing if the player SHOULD or SHALL reload the steering manifest after the specified TTL interval.

• In the semantics (table 6.3.1) the spec says:
  “Specifies how many seconds the client shall wait before reloading the DCSM.”
  => SHALL

• In section 7, bullet 7:
  “the client should parse it and retrieve the VERSION, TTL….”
  => SHOULD (it shall be SHALL)

• In section 7, bullet 8:
  “The client sets a timer to re-request the STEERING-SERVER-URL after TTL seconds.”
  => SHALL or SHOULD is missing

• In section 7, bullet 13:
  “It should still reload the RELOAD-URI after the specified TTL interval in case new service locations are added.”
  => SHOULD

2. Before clarifying if it must be SHALL or SHOULD, I’d like to consider the use case for which the DCSM could be refreshed before TTL interval.
   The recommended value for TTL is 300 seconds, and in some cases, it would be valuable to force the players to refresh the manifest without waiting for TTL interval.
   As an example, a new CDN server could be allocated and which we would like to prioritize as soon as possible. Instead of globally reducing the TTL to a very low value and overload the steering server, one could decide to force the players to update the DCSM when necessary.
   This would require obviously an external control mean to steer the players and I don’t know how and if this should be part of the content steering specification.
   As a control mechanism, one potential solution is for example to standardize a CMSD response header key to force the players to reload the DCSM.
   Whatever the solution to control the DCSM reloading, if that makes sense we should consider adding some text in specification to take into account the use case where client can reload the DCSM before waiting for TTL delay.

[haudiobe]

2023/10/06 TF meeting

• everyone please review the issue.
• how much do we want to make it a player implementation or mandate behaviour. Please discuss

[gwendalsimon]

2. Before clarifying if it must be SHALL or SHOULD, I’d like to consider the use case for which the DCSM could be refreshed before TTL interval.
   The recommended value for TTL is 300 seconds, and in some cases, it would be valuable to force the players to refresh the manifest without waiting for TTL interval.
   As an example, a new CDN server could be allocated and which we would like to prioritize as soon as possible. Instead of globally reducing the TTL to a very low value and overload the steering server, one could decide to force the players to update the DCSM when necessary.

I am not really convinced we need such a method to break the TTL (besides the complexity of it, see below). With a 300-sec TTL, all players are evenly distributed within this 5-min window, so they will come to the steering server one by one. In this particular use-case, it guarantees a graceful redirection of players to the new CDN, although forcing an update could generate a storm on the new CDN.

This would require obviously an external control mean to steer the players and I don’t know how and if this should be part of the content steering specification.

I would expect the steering server to be a stateless service, which does not store any information about the players. Furthermore, the steering server is not expected to know which players are still watching the session.

As a control mechanism, one potential solution is for example to standardize a CMSD response header key to force the players to reload the DCSM.

A CMSD message is issued by the CDN edge server. It cannot be the trigger to reload the DCSM since the decision to force the reload would come from the steering server... unless the steering server could ask the CDNs to send a CMSD message on its behalf.

[burak-kara]

As an example, a new CDN server could be allocated and which we would like to prioritize as soon as possible. Instead of globally reducing the TTL to a very low value and overload the steering server, one could decide to force the players to update the DCSM when necessary.

I remember this video from Apple WWDC22. They explain Pathway Cloning (starting from 8:16 with the background story) used to introduce a new CDN to the system. The idea still relies on the DCSM update at each TTL. They add PATHWAY-CLONES field to DCSM.

I try to illustrate the edge cases in which we want the new CDN to join the system before TTL (maybe preferably without any delay). But, for such cases, the player has the second (and so on) pathway on the PATHWAY-PRIORITY list as a backup.

[bbert]

Thanks @gwendalsimon and @burak-kara for your comments.

@burak-kara yes I know about pathway cloning but the use case was to update DCSM in order to get precisely new pathways before TTL delay.

@gwendalsimon I agree with you on the facts that steering server should preferably be stateless and the difficulties to ask CDN sending CMSD messages.

Let's tackle this issue in another way.
In fact the use case would be to enable a player to know about new pathways when it encounters some issues with current available pathways.

A potential solution is to complete the client bahaviour specification by adding the possibility for the client to refresh the DCSM when it encounters playback problems and when it has already switched to all of the available pathways.

By the way, I think we should explain more precisely in the client steering behaviour what is meant by "If the client encounters playback problems". When should a client make a BaseURL or Location switch?:

• when it gets an error from the server?
• when playback stalls after a failed request?
• when playback stucks on lowest bitrate?
• when time to first byte highly increase?
• others? (for example CMSD duress field from CDN)

Or is it completely opened to player implementation?
@dsilhavy do you have any opinion on that?

[haudiobe]

Encourage to review the latest specification here: https://members.dashif.org/wg/Interoperability/document/4810

[haudiobe]

We should check check what the IOP says. Do we reload the MPD in case of repeated segment 404? IOP and MPEG-DASH recommends to reload the MPD. That may resolve the issue for bertrand.

@dsilhavy please let know how you have implemented. Then we fix the spec. and the we check of bertrands still exists and then we fix the spec even more.

[bbert]

We should check check what the IOP says. Do we reload the MPD in case of repeated segment 404? IOP and MPEG-DASH recommends to reload the MPD. That may resolve the issue for bertrand.

And in case MPD uses the same baseUrl as for the segments, the player would not be able to refresh the MPD.
Please consider the use case where the player streams the content (MPD+segments) from a CDN and needs to be redirected to a newly created CDN/pathway to avoid playback failure.

[dsilhavy]

This is what dash.js does today:

• If we see a failed request, we trigger a clock sync to check if we have the right offset between client and server reference clock
• The number of retryAttempts and retryInterval can be configured via the player settings. Per default we try three times with a waiting time between 500-1000ms (depends on the type of the object that is requested)
• If the number of remaining retryAttempts reaches 0 we blacklist the BaseURL and move to the next available BaseURL.
• If all BaseURL elements are blacklisted playback is terminated.

As of today, we are not refreshing the manifest in case of repeated segment 404s. We are also not refreshing the DCSM.

What would be great if we can also collect the relevant parts of the specifications that dash.js shall implement to improve the current behavior.

[haudiobe]

Live TF 2024/03/01

Accepted that the spec details need to be collected.

[haudiobe]

IOP WG 2024/10/29

We suggest to update client behaviour

• in case of segment errors (404), an MPD update may provide new pathways and a content steering update should done as well.
• only after 3 attempts, an content steering update will be done. Should be done in the background.
• We still need to define the order in the player.

Please comment, we will update the spec.