[css-mediaqueries-5] Clarifications on [video-]dynamic-range MQs

[willcassella]

CSS Media Queries Level 5 introduced two new queries: dynamic-range and video-dynamic-range. Both have similar wording, and currently the wording for dynamic-range is like so:

dynamic-range represents the combination of max brightness, color depth, and contrast ratio that are supported by the UA and output device.

high
The combination of the User Agent and the output device fulfill all of the following criteria:

• it has a high peak brightness
• it has a high contrast ratio
• its color depth is greater than 24 bit or 8 bit per color component of RGB

standard

• One or more of the criteria for a high dynamic-range is not fulfilled.

The way Safari has implemented this media query is to make @media (dynamic-range: standard) checks always pass, and @media (dynamic-range: high) check against the capabilities of the current display device. This makes sense from a future-proofing perspective, as standard can be seen as a subset of high, and if ultra-high dynamic-range displays are ever introduced, high should be a subset of that.

However, the wording of the standard enum makes it sound as if standard and high should be mutually-exclusive. I think we should change this wording to reflect standard as a subset of high.

Additionally, the wording of "The combination of the User Agent and the output device..." has caused some confusion about what the capabilities of the user agent must be in order for the high query to pass. In the discussion surrounding the proposal for these queries, the idea was that this query would be a coarse representation of the display device's capabilities, and more detailed APIs such as MediaCapabilities could be used to gather specific information about the User Agent's capabilities for specific purposes. I think we should remove mention of the UA in the description of this media query to clarify this confusion.

[willcassella]

CC'ing participants in the original discussion for visibility: @tabatkins @gregwhitworth @astearns @svgeesus @Crissov @mwatson2 @AmeliaBR @chcunningham @vi-dot-cpp

[dbaron]

The other issue with the wording regarding "The combination of the User Agent and the output device..." is that User Agents may have different capabilities for CSS (and HTML) colors, colors in images, colors in canvas, colors in videos, etc. Which of these are required to be supported should probably be specified, both (separately) for dynamic-range and video-dynamic-range.

[nightpool]

As a developer, I agree that it would be kind of pointless to have (dynamic-range: high) return true if the actual user agent didn't support high dynamic range colors in at least some CSS-visible sense (CSS colors or colors in images). I think removing the mention of the UA in the description of this media query would mean that we end up speccing a property that is internally consistent, but isn't useful to developers and therefore doesn't gain adoption. And then in 12 months or so we'll be in the same place again and have to specify a new user-agent-video-dynamic-range media query.

[chcunningham]

In the discussion surrounding the proposal for these queries, the idea was that this query would be a coarse representation of the display device's capabilities, and more detailed APIs such as MediaCapabilities could be used to gather specific information about the User Agent's capabilities for specific purposes.

As a participant in the original discussion, I want to give a big +1 to ^this point. The dynamic-range MQ was motivated entirely by a need to describe the screen's capabilities. This is consistent with the overall purpose of Media Queries, which describe features (e.g. resolution) for the current media device (e.g. screen).

I think folks are confused by the video-dynamic-range feature. At a glance, this looks like it indicates when the UA is capable of playing HDR , so it follows that the non-video dynamic-range would correspond to non-video things like CSS. This is understandable but definitely not the intent.

Both video-dynamic-range and dynamic-range describe the screen. The video-dynamic-range proposal concerns a weird quirk of user agents implemented in TVs. Given limited system resources and a clear priority use case, these devices have a quirky implementation where video and non-video are rendered in distinct "planes", which amounts to distinct screens with distinct capabilities. Generally speaking, the video plane has a much higher resolution and dynamic range than that of non-video. Traditional user-agents have only one plane, such that video-dynamic-range and dynamic-range should behave identically.

Again, video-dynamic-range just describes the screen. It does not describe support for HDR video playback. This is a much more complicated question (w/ parameters like codec, transfer function, metadata, ...) which Media Queries is ill-suited to answer (it is instead answered by navigator.mediaCapabilities.decodingInfo()). IMO, questions about canvas, CSS, images, etc, should similarly follow this model of offering separate feature detection, choosing whatever method is most idiomatic for their respective domains.

[svgeesus]

Both video-dynamic-range and dynamic-range describe the screen. The video-dynamic-range proposal concerns a weird quirk of user agents implemented in TVs. Given limited system resources and a clear priority use case, these devices have a quirky implementation where video and non-video are rendered in distinct "planes", which amounts to distinct screens with distinct capabilities.

Yes, exactly. In other words, for most devices, these will have the same value. Only for a TV with a high-res, HDR video buffer and a (lower res, non-HDR) "graphics overlay buffer" will they have different values.

So the specification needs to more clearly reflect that.

[css-meeting-bot]

The CSS Working Group just discussed [css-mediaqueries-5] Clarifications on [video-]dynamic-range MQs.

The full IRC log of that discussion

Topic: [css-mediaqueries-5] Clarifications on [video-]dynamic-range MQs
github: https://github.com//issues/6793
florian: We have 2 MQ which are similar, dynamic-range and video-dynamic-range
florian: 2 issues in 1 here
florian: one is primarily editorial, doing poor job of explaining the difference between these two queries
florian: notion of video plane is fairly rare concept that relates to TVs and how they have unusual screen buffer impls that we can query separately
florian: core question raised in this issue, besides clarification
+1 to that editorial clarification
florian: is that we have 2 values: high and standard
florian: You might assume, and this apparently is what Safari did, that any device will match standard and some will match high as well
florian: this is not how currently specced, though
florian: devices expected to match one or the other
florian: not standard and high simultaneously
q+
florian: The query about color-gamut does behave differently, it has 3 values
florian: and these are additional, when you match broader gamut you also match narrower one
florian: unclear what we should do here
q?
ack chris
chris: I agree, media capability does the same thing, of using supersets
chris: overall I think it's a better model, think we can easily make spec say that
chris: the reason to do it is that when we add super-high later on, we want it to be a superset that still passes high
chris: so I think we should change the spec; not because it's what implemented, but because it makes more sense
dbaron: Wanted to add, think there's a 3rd issue
dbaron: currently wording in spec about combo of UA and output device
dbaron: needs clarification what the UA part of it means
see also https://w3c.github.io/media-capabilities/#hdrmetadatatype
dbaron: definition of high says, combination of UA and output device fulfill all the following criteria
dbaron: there are a number of ppl in the issue saying that the UA bit should be ignored, and should only be query about the output device
ack dbaron
dbaron: but if this is question of UA capability, which are you considering if UA can do for some things but not others
q?
dbaron: e.g. videos but not images, or ...
q+ to also mention HDR is opt-in and can change dynamically
florian: I suspect querying device and not UA is useless
florian: because if the UA isn't able to access the capability, it's not helping you
florian: but if capability varies within UA, that's a problem
Rossen_: Is one a superset of other? Is UA superset of device?
dbaron: I wouldn't think of it that way
florian: We're querying union of both
florian: need both UA and device to be capabe
chris: Also, HDR capability can vary over time
chris: high power usage
chris: so off by efault
chris: so need to know that it can change
chris: One, is the device capable of HDR mode, and then two, is it in that mode?
s/Is UA superset/Is UA subset/
chris: we are mixing those two up
florian: we do
ack chris
chris, you wanted to also mention HDR is opt-in and can change dynamically
ack fantasai
fantasai: we should take a resolution to change the query to superset model
fantasai: multiple things to be clarified
fantasai: let's ask editor to go back to the thread to work out results
fantasai: a lot of the clarifications are straightforward
fantasai: need a resolution to change the media queries
florian: Don't quite agree, question of capability or in HDR mode currently
florian: or videos vs images etc.
florian: not quite clarification, is change in functionality
florian: puzzle how to answer if we don't change the syntax
q+
florian: so can do things fantasai highlighted, but not enough to solve the issue
ack chris
chris: media capabilities spec talks about capability, "can the device do it"
chris: if we clarify that way, need the other query as well
chris: the capability and concrete "am I in this mode" are separate
florian: Is it expected to remain this way?
chris: especially for mobile, definite power drained
chris: can be switched automatically, doesn't require user action
chris: but there needs to be a capability
chris: and need to know which mode you're in
fantasai: i think we should spec as being able to use the switch
fantasai: from a media queries point of view, how are we styling the document - many of those questions on depend on current mode
fantasai: [missed parts]
florian: what do we do about if UA can do for images and video but not CSS colors?
florian: or some other variation?
florian: should we consider the UA does or does not fulfill the criteria?
florian: or do we think about query in a different way
florian: I haven't thought about deeply, but maybe you would have high-dynamic-range: video | canvas | etc.
florian: and would get something true if that specific thing is in HDR mode
Rossen_: sounds like a convesation to continue in GH
Rossen_: and doesn't seem we're ready to resolve just yet
Rossen_: propose we stop here, and then after working out proposal in issue, bring back and we'll record a resolution
Rossen_: but hopefully attracted some help on this issue

[frivoal]

Having had a little side discussion with the chrome folks, the tentative conclusion is that the way Safari implemented this is what we'd like the spec to align to. Specifically:

• standard is a subset of high, in the same vein as the color-gammut media feature values.
• Even though we just have two values, we do want to keep it as "standard" and "high", not turn it into "none" and something else so that it'd work in a boolean context, because in the future we may need to add one or more values to distinguish between high and even higher, depending on what sort of device gets made.
• This is about HDR capability, not about "is HDR active". The later question could possibly be interesting, but it's a more complicated one as that means different things on different types of devices, and the primary use of the MQ is just to gate-keep you entire HDR logic
• there is no author desire for sub-dividing the querry between hdr-for-video, hdr-for-images, hdr-for-css, etc, because while that would be more fine grained, it wouldn't actually be fined grained enough to answer all your hdr questions. You might still need to wonder about image/video formats, contrast ratios, quality of tone-mapping…… So it's not really a good idea to make the MQ more complex if it's not actually going to answer the nuances, and having it as a blunt question to know if you should bother with HDR at all seems to be what authors are actually calling for.
• There's nothing unusual about this querry when it comes to testing the capabilities of the output device vs that of the user agent vs that of the combination, so we should neither put special emphasis on it, nor go out of our way to claim that we're only testing one or the other.

In other words, a hypothetical different design, like

hdr:
    none | 
    [ 
        [img | img()] |
        [video | video()] |
        css |
        canvas
    ] &&
    active?

is not actually appealing to authors, and would add complexity for no practical reason, thus we should spec what Safari is shipping.

[frivoal]

Misconfigured the log bot while discussing this. The discussion is at https://logs.csswg.org/irc.w3.org/css/2021-12-15/#e1452098, and the resolution was :

RESOLVED: dynamic-range: standard also matches when dynamic-range: high matches