Move LOC header metadata from object header extensions to object payload

[mzanaty]

In the Toronto interim, there was strong support to move the LOC header metadata from object header extensions to object payload using a payload header specific to LOC. LOC metadata would become opaque to relays, and perhaps encrypted end-to-end in schemes such as Secure Objects. People generally liked those consequences.

If any LOC metadata must be accessible by relays, it must be registered in the MOQT Object Header Extensions registry, perhaps in addition to the LOC metadata registry if it must be inaccessible to relays in some uses but accessible in others.

[suhasHere]

I am not sure If i recall a strong support in that direction of moving all LOC extensions to object payload. There are few that needs to be immutable and some that needs to be private.

Secure objects is defining Private Extensions now and LOC should use it to for things that needs to be end to end encrypted: https://github.com/suhasHere/moq-secure-objects/blob/main/draft-jennings-moq-secure-objects.md#private-extension-pvt-ext

Please note, these extensions are still registered as MOQ Header Registry entries.

[yekuiwang]

What I remember is that the architecture shown in slide #7 of the LOC presentation at the meeting, which is what is in LOC v-01, should be kept, where LOC Header Extensions map to MOQ Header Extensions. The architecture shown in slide #8, to make LOC Header Extensions as MOQ payload, was not agreed.

[suhasHere]

+1 to Yekui . that's exactly what I recall as well.

[mzanaty]

See the Toronto notes. Mo, Christian, Cullen, Victor, Luke preferred v2. YK preferred v1. Suhas said needs review. YK, Suhas, please explain your positions or any alternate proposals. Otherwise it appears there is rough consensus for v2. Since it’s so close to the draft deadline, I won’t publish v2 yet so we can discuss this in Montreal and hopefully resolve during the session.

[yekuiwang]

See the Toronto notes. Mo, Christian, Cullen, Victor, Luke preferred v2. YK preferred v1. Suhas said needs review. YK, Suhas, please explain your positions or any alternate proposals. Otherwise it appears there is rough consensus for v2. Since it’s so close to the draft deadline, I won’t publish v2 yet so we can discuss this in Montreal and hopefully resolve during the session.

Where is that notes? That does not align with my recollection at all.

[mzanaty]

https://notes.ietf.org/notes-ietf-interim-2025-moq-22-moq

[yekuiwang]

I don't see Victor's comment (Victor: If things are encrypted, should go in payload, if not in immutable headers) and Luke's comments (Luke: Comes down to layering. LOC extensions don’t neccesarily need to be relay visible, but if a header is intended to target relay behavior it should be in MoQ header; Luke: Should create MoQ headers and application header block and define separately. Luke: I like the proposed design, keep application data isolated to object payload.) as clearly preferred v2.

And note that Alan and Ian also had a comments: "Alan: Can we table this and discuss with concrete proposal." "Ian: For SMAF decision, do whatever is most efficent. We can wrap on client side to make migration/compatibility easier."

Overall, I don't see there was a consensus to move on with making the LOC Header Extensions mapping to MOQ payload instead of MOQ Header Extensions. At the meeting, I commented that the only good reason of having LOC Header Extensions is for carrying metadata information that is helpful for transmission optimization, as whatever metadata information that is not helpful transmission optimization but could be helpful for media decoding post-processing or other application functionalities should be left for video coding specifications and other application specifications, they should not be in the scope of MoQ family of standards, including LOC. These comments were not captured, but when I made the comments, I clearly remembered that people were nodding.

In any case, maybe we should check with people to see whether more people thought there was a consensus with making the LOC Header Extensions mapping to MOQ payload instead of MOQ Header Extensions.

@vasilvv @kixelated @afrind @ianswett @wilaw

[kixelated]

If metadata is not intended for a relay, then it should be part of the opaque object payload (and potentially encrypted). That's definitely the case for LOC headers in the current draft.

[wilaw]

From my side, I thought there was consensus that we should move the LOC metadata (such as timestamp) to the MOQ Object payload a.k.a v2 of the design. This metadata is not read by relays, is not required for transmission or routing and is intended only for the final subscriber. In my mind there is a very architectural litmus test for whether information should be placed in an extension header or the payload. If the relays are going to act upon the data (such as a filter operation), then it should be in an extension otherwise its should be in the payload. The LOC headers (capture timestamp, video frame marking), fail that test and therefore should be in the payload. The only header that perhaps should be in exposed in an extension header might be Audio Level IF a relay filter operation is going to be applied (i.e give me the 5 most active audio feeds).

We have had this discussion at multiple interims now and it feels circular. Each time it seems there is agreement in the room and then afterwards one or two people say there isn't and then we go back in the loop. I'd ask the chairs to please have a poll so we can count votes and finalize the decision.

[yekuiwang]

Thanks to @kixelated and @wilaw for clarifying your opinions. To me, the following rule is good:

If the relays are going to act upon the data (such as a filter operation), then it should be in an extension otherwise its should be in the payload.

Regarding video frame marking, I remember @mzanaty said at the Toronto meeting that would be removed (Mo please correct me if I got you wrong on this aspect).

Regarding Audio Level (and whatever other information, e.g., that might be proposed in the future) that is useful for relays to see, it might worth thinking on what is the best way of conveying the information, e.g., in an an Object Extension Header, or as a Parameter in a MOQT control message, as discussed in moq-wg/moq-transport#1316 for the media bitrate information (or the the expected time, as @gwendalsimon suggested).