Support of a start-code-based format in LOC

[yekuiwang]

In the current LOC draft, for video, a start-code-based format for H.264/AVC and H.265/HEVC video is supported. As we know, in H.264/AVC and H.265/HEVC, the start codes based Annex B bitstream format was included only for use with bit-oriented systems in particular MPEG-2 TS.

Do we think that there is a possibility, when MoQ is deployed, MoQ might be used together with MPEG-2 TS, or use of some content stored from MPEG-2 TS broadcast environment needs to be supported?

My feeling is that is very unlikely. If people agree with this feeling, then support of a start-code-based format might not worth the effort and thus it might be cleaner to drop the support of that format.

[yekuiwang]

@mzanaty Mo, what do you think about the above suggestion?

[mzanaty]

WebCodecs EncodedVideoChunk supports "annexb" format which uses start codes. I think LOC should continue to support this for compatibility with WebCodecs not MPEG-2 TS. It's very simple to support, so I see no benefit to remove this, which is more effort than keeping it.

[yekuiwang]

OK. However, note that the suggestion is not asking for compatibility with MPEG-2 TS, but the opposite, to not support the video format that is only needed for compatibility with MPEG-2 TS.

[tobbee]

I think that compatibility with Web Codecs is an argument, but I would have preferred that Web Codecs did not support Annex B.

Looking a bit ahead: One reason to avoid support for Annex B is when encryption is added. In that case, the encryption may generate false start codes that cannot be distringuished from other start codes. This is the case in HLS Sample Encryption where an extra round of start-code-emulation-prevention byte insertion is done on the NALU payload after encryption. This makes the total frame length increase with an amount that cannot be predicted since it depends on the encryption key and content. Length-based NALU packaging does not have this issue, so for HLS using fMP4 the sample size does not depend on encryption.

Therefore, at least in the future when introducing encryption support in LOC, Annex B should not be supported, in my view.
Dropping Annex B support already now would mean a slight advantage for the case where one wants to terminate LOC in the receiver and translate into mp4 fragments and use MSE playback rather than web codecs.