vo_gpu_next: linearize sRGB as a pure 2.2 power function

[kasper93]

The sRGB EOTF is a pure gamma 2.2 function. There is some disagreement regarding the sRGB specification and whether it should be treated as a piecewise function. Many displays are actually gamma 2.2, and content mastered for PC is typically affected by that. Therefore, linearize it as such to avoid raised blacks.

See:
IEC 61966-2-1-1999
https://community.acescentral.com/t/srgb-piece-wise-eotf-vs-pure-gamma/4024
KhronosGroup/DataFormat#19
https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/12
https://github.com/dylanraga/win11hdr-srgb-to-gamma2.2-icm

[github-actions]

Download the artifacts for this pull request:

Windows

• mpv-aarch64-pc-windows-msvc
• mpv-i686-w64-mingw32
• mpv-x86_64-pc-windows-msvc
• mpv-x86_64-w64-mingw32

macOS

• mpv-macos-13-intel
• mpv-macos-14-arm
• mpv-macos-15-arm

[Headcrabed]

@kasper93 What about enable this on ACM-disabled device by default?

Another read material that could be added into documentation: https://projects.blender.org/blender/blender/issues/145022

[na-na-hi]

The sRGB EOTF is a pure gamma 2.2 function. There is some disagreement regarding the sRGB specification and whether it should be treated as a piecewise function.

I am not seeing any disagreement here. The sRGB specification is very clear about this:

• The sRGB reference display has an EOTF of a pure gamma 2.2 function. This is for the reference display ONLY - it is intended to be used by the sRGB reference display to map the sRGB encoded values to light intensity.
• The piecewise function is used for everything else: converting between sRGB and other color representations like XYZ, which requires the function to be invertible.

The above indicate that the "gamma 2.2" treatment should only be explicitly done during the final output phase when displaying sRGB content on an HDR display (to keep the light intensity the same as on sRGB display). For sRGB display which accepts sRGB-encoded values, no conversion should be done. The source should also not be treated as encoded in gamma 2.2 in any situation.

[Headcrabed]

The above indicate that the "gamma 2.2" treatment should only be explicitly done during the final output phase when displaying sRGB content on an HDR display (to keep the light intensity the same as on sRGB display). For sRGB display which accepts sRGB-encoded values, no conversion should be done. The source should also not be treated as encoded in gamma 2.2 in any situation.

@na-na-hi Similar discussions happened on every link provided above.... Please read some of them.

[kasper93]

@na-na-hi: I provided 4 links with discussions that should explain to you why this option exists and why it is useful when linearizing sRGB to encode in PQ. (also this is a draft for a reason...)

[na-na-hi]

@na-na-hi Similar discussions happened on every link provided above.... Please read some of them.

I read through these links, and everyone there is confused by the ambiguous use of terms "EOTF" and "OETF". This is the main reason why they cannot comprehend that both of my points can be correct at the same time, and have to declare one of them wrong to resolve the apparent conflict.

Even Jack Holm, who is mostly correct on this issue, is confused: In his email he said "The sRGB standard does not specify an OETF", while at the same time saying that sRGB->XYZ is specified using a two part EOTF. However, both sRGB->XYZ and XYZ->sRGB conversion formulas are defined, which means that if the conversion specifies EOTF, it must also specify OETF, which conflicts with his other statement.

By applying the strict standard regarding these terms to only refer conversions between real, physical light and digital encodings, it can be concluded that the sRGB standard indeed does not specify an OETF. It only specifies a gamma 2.2 EOTF. The sRGB->XYZ and XYZ->sRGB conversion is only between digital values and has no bearing on OETF/EOTF in a strict sense.

He is also wrong on the gamma 2.2 being an "approximation": nowhere is the standard says it is an approximation, and the gamma 2.2 formula is presented in an unambiguous way. In fact, it refers to Annex A, which directly rejects the usage of ambiguous "gamma" term, suggesting the intention of an accurate power law representation. sobotka also argued why it is the case (although also with confusion about EOTF/OETF terminology). This is easier to understand considering that the sRGB->XYZ conversion is NOT an EOTF. The purpose is to convert sRGB into other colorspace specifications, and that colorspace is converted to display using the appropriate display EOTF.

@na-na-hi: I provided 4 links with discussions that should explain to you why this option exists and why it is useful when linearizing sRGB to encode in PQ

But I did acknowledge the case for mapping sRGB for a target PQ display (and I think this is the correct thing to do, since mpv takes the role of sRGB reference display (exact power 2.2 EOTF) here to map sRGB to light intensity). What is the problem here?

[kasper93]

But I did acknowledge the case for mapping sRGB for a target PQ display (and I think this is the correct thing to do, since mpv takes the role of sRGB reference display (exact power 2.2 EOTF) here to map sRGB to light intensity). What is the problem here?

Sorry, it's probably me misunderstanding.

The source should also not be treated as encoded in gamma 2.2 in any situation.

I this this was what prompted my reply. There is still a bit of engineering issue on our color management side. Because depending on input and output transfer it need to decide different things. But similarly as with bt.1886 where we use inverse of it to linearize source, I think we can use gamma 2.2 to linearize sRGB. This way we are working in display space linear light. Now the engineering part I mention is to infer what transfer function use to delinearize. For example if target-trc=srgb we would delinearize with gamma2.2 to preserve gamma. For target-trc=pq we would use directly display refered linear light to convert to luminance values. Targeting linear should also be fine, because it should be encoded for display response by compositor, so our "environment encoding" can stay in it.

Does it make sense, if something is not clear let me know.

EDIT: Of course we could also do sRGB decoding and convert to display light later, but I think it effectively would be the same, for our processing sake that we do in linear light. (and hopefully we only linearize / delinearize once)
EDIT2: I will also remove this option, because it's not needed in fact.

[na-na-hi]

But similarly as with bt.1886 where we use inverse of it to linearize source, I think we can use gamma 2.2 to linearize sRGB. This way we are working in display space linear light.

This is the incorrect thing to do according to sRGB standard, because you are linearizing to another colorspace for further processing on the linear light intensity values (scaling, shader, etc) and not to a reference display, so it needs to use the piecewise function, not the gamma 2.2 function.

Of course we could also do sRGB decoding and convert to display light later, but I think it effectively would be the same, for our processing sake that we do in linear light.

It is not the same. In general f(g(x)) is different from g(f(x)). Processing should be done in absolute intensity, not display intensity.

[llyyr]

This is the incorrect thing to do according to sRGB standard, because you are linearizing to another colorspace for further processing on the linear light intensity values (scaling, shader, etc) and not to a reference display, so it needs to use the piecewise function, not the gamma 2.2 function.

Could you point out what section of the spec supports this interpretation? If I understand you correctly, mpv linearizing with power2.2 function as the EOTF would be correct if it were to output sRGB/gamma2.2*, but it would not be correct to do so if mpv were to output some other colorspace, say bt.1886?

*I say sRGB/gamma2.2 but they have the same EOTF function so they should be the same.

[kasper93]

But similarly as with bt.1886 where we use inverse of it to linearize source, I think we can use gamma 2.2 to linearize sRGB. This way we are working in display space linear light.

This is the incorrect thing to do according to sRGB standard, because you are linearizing to another colorspace for further processing on the linear light intensity values (scaling, shader, etc) and not to a reference display, so it needs to use the piecewise function, not the gamma 2.2 function.

Of course we could also do sRGB decoding and convert to display light later, but I think it effectively would be the same, for our processing sake that we do in linear light.

It is not the same. In general f(g(x)) is different from g(f(x)). Processing should be done in absolute intensity, not display intensity.

Well of course, the whole standard is about this.

And yet, this is what is required to preserve perceptual look of the image as it is intended to be viewed. Similarly we don't linearize bt.709 with inverse of camera oetf and instead use bt.1886.

Our rendering pipeline internally is in display intensities (not encoding ones), HDR like. Also with BPC already applied.

This is the incorrect thing to do according to sRGB standard, because you are linearizing to another colorspace for further processing on the linear light intensity values (scaling, shader, etc) and not to a reference display, so it needs to use the piecewise function, not the gamma 2.2 function.

Could you point out what section of the spec supports this interpretation? If I understand you correctly, mpv linearizing with power2.2 function as the EOTF would be correct if it were to output sRGB/gamma2.2*, but it would not be correct to do so if mpv were to output some other colorspace, say bt.1886?

*I say sRGB/gamma2.2 but they have the same EOTF function so they should be the same.

This is what is defined by sRGB standard and the "mismatch" is discussed in introduction

[llyyr]

This is what is defined by sRGB standard and the "mismatch" is discussed in introduction

This doesn't explain why it would be incorrect to use power2.2 EOTF if mpv were to output bt.1886 or PQ. (I'm still waiting for nanahi to say that this is what they meant)

From my understanding, such an operation would be as follows:

srgb video -> mpv linearizes using power2.2 -> work in linear light -> mpv delinearizes to pq or bt.1886 or whatever.

From my understanding, nanahi is saying that mpv linearizes using power2.2 step is wrong if mpv would in the end delinearize to any colorspace except sRGB.

[kasper93]

(I'm still waiting for nanahi to say that this is what they meant)

Ok, sorry. I won't interrupt you anymore.

[sobotka]

It is not the same. In general f(g(x)) is different from g(f(x)). Processing should be done in absolute intensity, not display intensity.

As folks are grasping the nuance here, it becomes a question as to what the “correct” composition operation would be.

We can see that if we consider the more obvious case of BT.709 to BT.1886, it is the operator, aka the display transfer characteristic and colourimetry, that determines the relative wattages of the presented stimuli. It would appear be unsound advice to suggest that compositing the presented stimuli of window A over window B should be decoded to relative wattages using the inverse of the BT.709 encoding characteristic; if the goal were to be to emulate a composition as though the planes of the windows were physicalist / materialist panels, it is strictly the presented stimuli relative wattages that matter here.

Given that the operator drives the meaning of the data encoding, we can see some sense for considering the PQ or HLG EOTF as driving the compositing of the presentation mechanisms. If we remove the composition out of the equation, the normative presentation of an sRGB two part encoding is a vanilla 2.2 EOTF, and to draw an equivalent presentation, the relative wattages should align. Therefore the presenting stimuli state would logically be the appropriate encoding for the output data state operation; placing the sRGB encoding into the PQ / HLG encoded state as though it were being emitted from a 2.2 EOTF presenting medium.

It follows that if we agree the presentation state is the driving function, then as with the BT.1886 example, the normative 2.2 operator as decoding for presented relative wattages should be employed for composition.

[na-na-hi]

This doesn't explain why it would be incorrect to use power2.2 EOTF if mpv were to output bt.1886 or PQ. (I'm still waiting for nanahi to say that this is what they meant)

From my understanding, such an operation would be as follows:

srgb video -> mpv linearizes using power2.2 -> work in linear light -> mpv delinearizes to pq or bt.1886 or whatever.

From my understanding, nanahi is saying that mpv linearizes using power2.2 step is wrong if mpv would in the end delinearize to any colorspace except sRGB.

PQ specifies the displayed luminance. It is different from the absolute luminance level converted from sRGB using the piecewise function.

If mpv is outputing to PQ the correct pipeline should look like this:

sRGB video -> mpv linearizes using sRGB piecewise function -> work in absolute linear light -> processed absolute linear light -> mpv delinearizes using sRGB piecewise function -> processed sRGB video -> mpv linearizes to display light using power2.2 -> mpv delinearizes to pq

Note that the "processed absolute linear light -> mpv delinearizes using sRGB piecewise function -> processed sRGB video -> mpv linearizes to display light using power2.2" step can be a single "absolute linear light -> display linear light" step if we precompute the relationship.

So what this PR should do is the following: when source is sRGB and target display does not accept sRGB encoded values, add the "absolute linear light -> display linear light" step before converting from linear colorspace to output colorspace.

[Headcrabed]

@kasper93 By setting to "Both" by default, you are breaking ACM support....

[kasper93]

@kasper93 By setting to "Both" by default, you are breaking ACM support....

By not setting both by default, we are breaking Wayland support. I'm aware of that, and frankly you should be using PQ output from mpv if your display is in HDR mode.

[sobotka]

sRGB video -> mpv linearizes using sRGB piecewise function -> work in absolute linear light -> processed absolute linear light -> mpv delinearizes using sRGB piecewise function -> processed sRGB video -> mpv linearizes to display light using power2.2 -> mpv delinearizes to pq

It is not logical to suggest that the direct to PQ relative stimuli wattages should be power 2.2, and that the compositing of two “layers” be calculated differently, using the two part decoding.

If we consider two buffers A and B, which are “unoccluded” and presented under the 2.2 power function simultaneously on the display medium, the relative “linear” wattages follow from the 2.2 power function for both regions.

If we attempt to emulate moving A over B, and occluding B by some degree of A for an “overlapping window” emulation, it is logical that the as presented relative wattages derived from the 2.2 power function should be employed. The measured energy at the faceplate would indicate the power 2.2 output, not the two part input.

[Headcrabed]

@kasper93 By setting to "Both" by default, you are breaking ACM support....

By not setting both by default, we are breaking Wayland support. I'm aware of that, and frankly you should be using PQ output from mpv if your display is in HDR mode.

No it won't break. haasn/libplacebo@4d4938d would make it choose linear color space on Linux and macOS. Only windows ACM uses VK_COLOR_SPACE_SRGB_NONLINEAR_KHR.

[Headcrabed]

Also seems on some devices, SDR 10bit out would only work when ACM is enabled... And PQ won't help here.

[llyyr]

haasn/libplacebo@4d4938d would make it choose linear color space on Linux and macOS

FWIW linear output is wrong unless you set --target-contrast=inf... except the option is also buggy on mpv and doesn't have the same result as in plplay (if you set min_luma to the 1e-6 minimum). I will look into this sometime soon when I have the time

[Headcrabed]

haasn/libplacebo@4d4938d would make it choose linear color space on Linux and macOS

FWIW linear output is wrong unless you set --target-contrast=inf... except the option is also buggy on mpv and doesn't have the same result as in plplay (if you set min_luma to the 1e-6 minimum). I will look into this sometime soon when I have the time

But we don't have any other better choices... Linear, sRGB and linear scRGB are the only choices on some Wayland compositors, and some of them just implement sRGB piece-wise as 2.2.

[mahkoh]

RE the rest of the discussion here:

One of the goals of the wayland color management protocol (possibly the goal) is to have the same content look the same regardless of the color space conversions performed, as long as the final and all intermediate color spaces can correctly represent the content. For example, consider the following two pipelines:

Pipeline 1:

1. mpv plays sRGB video
2. mpv sends the video to the compositor using sRGB color description
3. compositor converts to PQ
4. compositor displays on HDR10 display

Pipeline 2:

1. mpv plays sRGB video
2. mpv converts to PQ
3. mpv sends the video to the compositor using PQ color description
4. compositor displays on HDR10 display

To achieve the goal, mpv and the compositor must agree how to convert sRGB to PQ. If compositors always linearize sRGB content using gamma22, mpv has to do the same if it wants to achieve the same goal. Whether or not that is defined in any standard other than wayland is irrelevant.

[Headcrabed]

RE the rest of the discussion here:

One of the goals of the wayland color management protocol (possibly the goal) is to have the same content look the same regardless of the color space conversions performed, as long as the final and all intermediate color spaces can correctly represent the content. For example, consider the following two pipelines:

Pipeline 1:

1. mpv plays sRGB video

2. mpv sends the video to the compositor using sRGB color description

3. compositor converts to PQ

4. compositor displays on HDR10 display

Pipeline 2:

1. mpv plays sRGB video

2. mpv converts to PQ

3. mpv sends the video to the compositor using PQ color description

4. compositor displays on HDR10 display

To achieve the goal, mpv and the compositor must agree how to convert sRGB to PQ. If compositors always linearize sRGB content using gamma22, mpv has to do the same if it wants to achieve the same goal. Whether or not that is defined in any standard other than wayland is irrelevant.

And that's why I made this commit. haasn/libplacebo@4d4938d

[kasper93]

To achieve the goal, mpv and the compositor must agree how to convert sRGB to PQ. If compositors always linearize sRGB content using gamma22, mpv has to do the same if it wants to achieve the same goal. Whether or not that is defined in any standard other than wayland is irrelevant.

Yes, that's clear. Currently mpv agrees with what Windows does. This PR will allow to match Wayland behavior. I like the platform as a default, which will work on both platforms and frankly avoids the headache of deciding what should be the default. Smarter people already decided.

And that's why I made this commit. haasn/libplacebo@4d4938d

Linear output is currently not supported on Windows. We don't implement scRGB which would be possible to use on Windows. Also PQ output from mpv should be compatible with whatever compositors are doing, unless compositors are wrong, but that's why this option can be adjusted by user to decide for themselves.

[Headcrabed]

Linear output is currently not supported on Windows. We don't implement scRGB which would be possible to use on Windows. Also PQ output from mpv should be compatible with whatever compositors are doing, unless compositors are wrong, but that's why this option can be adjusted by user to decide for themselves.

Use PQ for SDR is not a good choice. More dynamic range means possibly color banding.

But if the platform option could detect what OS that user is using and also detect whether ACM is enabled then there's no problem to use sRGB on Windows.

[Headcrabed]

To achieve the goal, mpv and the compositor must agree how to convert sRGB to PQ. If compositors always linearize sRGB content using gamma22, mpv has to do the same if it wants to achieve the same goal. Whether or not that is defined in any standard other than wayland is irrelevant.

Yes, that's clear. Currently mpv agrees with what Windows does. This PR will allow to match Wayland behavior. I like the platform as a default, which will work on both platforms and frankly avoids the headache of deciding what should be the default. Smarter people already decided.

Also do we really need this option? On Wayland and macOS we always use linear, and on windows, ACM uses piece-wise, only available case for this option is ACM-disabled windows/X11/Wayland without color management support. But in these cases, the only method to output correct color is to provide MPV with ICC file. But ICC file has the actual TRC curve for the monitor, so we still don't need this option.

[llyyr]

On Wayland and macOS we always use linear

That is only for Vulkan, not opengl. Also this is configurable, and I definitely won't let the default stay at linear unless we also set target-contrast=inf by default. (provided the option actually works)

[na-na-hi]

But in these cases, the only method to output correct color is to provide MPV with ICC file.

Displays without colorspace info or ICC are assumed standard sRGB display (gamma 2.2). (Annex C.2) As long as the display is standard sRGB, the color is correct.

[Headcrabed]

On Wayland and macOS we always use linear

That is only for Vulkan, not opengl. Also this is configurable, and I definitely won't let the default stay at linear unless we also set target-contrast=inf by default. (provided the option actually works)

OK. Then I guess we need to improve target-contrast’s auto implementation to check os/api/ACM status?

[kasper93]

To achieve the goal, mpv and the compositor must agree how to convert sRGB to PQ. If compositors always linearize sRGB content using gamma22, mpv has to do the same if it wants to achieve the same goal. Whether or not that is defined in any standard other than wayland is irrelevant.

After consideration, I'm not sure the goal of mpv should be the same. While I agree in perfect world mpv and compositors would agree on this conversion. In fact I don't see why PQ output from mpv should be platform specific. mpv aims to provide best fidelity output and consistent regardless of platform used. This also applies to scaling and tonemapping, which is applied depending on the target parameters and current source. And the same goes for sRGB to PQ conversion.

Otherwise the whole rendering pipeline in mpv is pointless and we should just use direct output to dmabuf_wayland.

I'd even say that mpv's job is to solve the limitations of the compositors. Consistent platform independent output, not platform dependent. If it means that mpv will look different than Windows Media Player, so be it.

Also in classic mpv fashion, everything is configurable, so as a user you can tune the output however you like.

[mahkoh]

If pipelines 1 and 2 produce different output, then the output is not platform independent because mpv chooses the pipeline based on platform preferences and capabilities.

[kasper93]

If pipelines 1 and 2 produce different output, then the output is not platform independent because mpv chooses the pipeline based on platform preferences and capabilities.

Pipeline 1 and 2 produce the same output on any platform, that's the goal.

[mahkoh]

Explain.

[kasper93]

Explain.

Which part exactly? I'm not aligning sRGB to PQ conversion to be platform specific, instead I "fix" Windows conversion by pre-applying G2.2 EOTF when output is HDR and our swapchain is sRGB.

[mahkoh]

Which part exactly?

You're saying that P1 and P2 produce the same output and you're also saying that the PQ signal produced by mpv should not depend on the compositor.

However, if step 3 in P1 depends on the compositor, then step 2 in P2 must also depend on the compositor to produce the same observable output.

[kasper93]

Which part exactly?

You're saying that P1 and P2 produce the same output and you're also saying that the PQ signal produced by mpv should not depend on the compositor.

However, if step 3 in P1 depends on the compositor, then step 2 in P2 must also depend on the compositor to produce the same observable output.

P2 doesn't depend on compositor. This conversion is the same regardless of platform.
P1 depends on compositor. And would look like so:

Pipeline 1:

1. mpv plays sRGB video
2. (optionaly, on Windows) mpv pre-apply G2.2 EOTF (linearize with G2.2, delinearize with sRGB)
3. mpv sends the video to the compositor using sRGB color description
4. compositor converts to PQ
5. compositor displays on HDR10 display

[kasper93]

Updated PR. This is the version that works mostly correct in each case.

For everyone’s information regarding the sRGB EOTF. I think we can acknowledge that many displays actually implement piecewise sRGB EOTF, especially in "srgb mode" if such a mode is provided. Of course, it may mean only primaries, but from what I have seen, monitors seems to implement sRGB piecewise. Now here is the issue. Sending sRGB encoded signal to gamma2.2 display is looking fine, after all this is what the spec prescribed. However sending gamma2.2 to sRGB (piecewise) display is looking quite bad, blacks are boosted to the point it's unwatchable. And you can argue that such display is not spec compliant, but reality is different as we know. Jest letting you know, that it may be not as pragmatic as you think to be technically correct here.