Video Content Description

[haudiobe]

For the video content description in 4.2, would it not be more advantageous to use existing signaling and code points, for example on what is used in Codec-Independent code points. We have done some video signal description also in 3GPP TS 26.265 https://ftp.3gpp.org//Specs/archive/26_series/26.265/26265-050.zip - it would be good to have a harmonized set of code points that are also extensible without having to update the specification, for example if we have a new codec or format.

[haudiobe]

There is also media capabilities API in W3C

[haudiobe]

Here are some references

• https://mp4ra.org/registered-types/codecs
• https://www.w3.org/TR/media-capabilities/#hdrmetadatatype
• https://www.itu.int/rec/recommendation.asp?lang=en&parent=T-REC-H.273-202407-I

Alternative would be to register this somehow.

[GwenSynamedia]

TLDR;

The watermarking API currently uses a set of enums to define the characteristics of the input video to be processed:

1. §4.2.1 raster format (based on ffmeg command line)
2. §4.2.2 interlacing (ad-hoc enum)
3. §4.2.4 static HDR configuration (ad-hoc enum)
4. §4.2.5 video codec (ad-hoc enum)

Dynamic HDR configuration settings are covered on a per-frame basis in §4.3.8 (in a very lose ad-hoc way).

Rather than relying on ad-hoc enums that may need to be updated in the future, @haudiobe is enquiring if we could use any standard description signaling. The question to video encoders people that will reach this thread is whether standardized signaling exists for such things at encoder API level.

Current enums from the watermarking spec

Interlacing

typedef enum {
  FF_INTERLACED_TOP_FIRST    = 0, // Interlaced format with odd/even indication
  FF_INTERLACED_BOTTOM_FIRST = 1, // Interlaced format with odd/even indication
  FF_PROG                    = 2, // Progressive mode
  FF_3DSBS                   = 3, // 3-D Side-by-Side
  FF_3DTB                    = 4  // 3-D Top-Bottom
} wmp_format_t;

High-dynamic range

typedef enum {
  SDR             = 1,
  HDR10           = 2,
  HDR10PLUS       = 3,
  HLG             = 4,
  DOLBYVISION     = 5,
  SL_HDR_1        = 6,
  SL_HDR_2        = 7,
  SL_HDR_3        = 8,
  PQ10            = 9,
  HDR_VIVID       = 10,
  range_undefined = 99
} wmp_hdr_mode_t;

Video codec

typedef enum {
  RAW             = 0,
  AVC             = 1,
  HEVC            = 2,
  AV1             = 3,
  MPEG4           = 4,
  VP9             = 5,
  VP10            = 6,
  VVC             = 7,
  EVC             = 8,
  LCEVC           = 9,
  MPEG2           = 10,
  AVS3            = 11,
  codec_undefined = 99
} wmp_codec_t;

Personal opinion

This point has been discussed on SVTA / DASH IF calls on Feb. 12th and Feb 26th 2025.
The discussion did not really converged.

Personally, I would love to refer to existing signaling and code points.
However, after briefly reviewing the suggested references, none of them seem to be really fitting the bill.
For instance, mp4ra defines nice 4ccs that would be great to reuse, but:

1. it does not incorporate HDR consideration except for DolbyVision (dav1, dva1, dvav, dvh1, dvhe) i.e. WMP will still need to define code points for other HDR modes;
2. it declares various packaging variants for the same codec, eg. avc1, avc2, avc3, avc4 for AVC whereas WMP really only needs the coded to know how to initialize its bitstream parser for post-encode watermarking solution.

I am sure that we can make it fit (I am not hard stuck on interlacing | hdr | codec separation) , but I feel like the added complexity seems to counterbalance the benefit of not defining our ad-hoc enums.

[GwenSynamedia]

The command ffmpeg -codecs defines the following codes (would be in line with raster format)

RAW  > rawvideo
AVC   > avc
HEVC  > hevc
AV1   > av1
MPEG4 > mpeg4
VP9  > vp9
VP10
VVC  > vvc
EVC
LCEVC
MPEG2 > mpeg2video
AVS3  > avs3

Currently no correspondance for VP10, EVC, and LCEVC.

[HegarAWS]

GwenSynamedia did a nice job summarizing and I agree with his sensibilities. I haven't studied the problem enough to understand exactly which enums are needed by the watermarking API, but (as a software engineer that works on MediaLive) I don't think it will cause any great burden for encoder vendors to conform to watermarking specific API enums. Having the enums in the watermarking API requires that the API evolve as new formats need to be added over time, but that also gives an opportunity for the API authors to clearly indicate which are supported and differentiate between required and optional.