[css-images-3] change which images image-orientation applies to

[faceless2]

In #4165 it was resolved that EXIF orientation applies to all images by default. There was some discussion about which images the image-orientation property and talk of a resolution, but no actual resolution.

This matters because we've got two sets of WPT tests added in the last six months which disagree - see web-platform-tests/wpt#18549 (comment) - and different results in Firefox and Chrome (see a, b). So we need a once-and-for-all resolution on this.

@heycam made the same observation in #4165 (comment), and we'd like to implement this but can't until it's clarified.

So the question is: does the image-orientation property apply to "decorative" images, such as background-image and border-image? And if it does not (as currently specified), the second question is can we tightly define "decorative" images.

@zcorpan made a full list at web-platform-tests/wpt#18549 (comment)

CSS

• ::before/::after + content
• background-image
• list-style-image
• border-image
• cursor

SVG

• image

HTML

• img
• canvas drawImage() + getImageData()
• video poster
• input type=image
• embed
• object

load an image in an iframe
load an image in a top-level doc (this already respects EXIF in all browsers I believe)
Favicon link rel=icon / favicon.ico (manual tests?)
web app manifest icon (manual test?)

• I would expect "content" images to be those displayed from content, list-style-image (which generates content), SVG image, HTMLimg, embed, object, and video poster. The image-orientation property should apply to all of these.
• I would expect "decorative" images to be background-image, border-image and mask-image.
• image-orientation does not apply to cursor, top-level docs, favicon or web-app manifests.
• I'm slightly unsure about exactly where iframe fits.

cc @schenney-chromium

[heycam]

So the question is: does the image-orientation property apply to "decorative" images, such as background-image and border-image? And if it does not (as currently specified), the second question is can we tightly define "decorative" images.

Agree that that's what we need to answer.

The goal of the image-orientation property is to make it easy to turn off the new automatic re-orientation behavior, in case authors have content that breaks with the new behavior. (We had a few bug reports in Firefox of content that broke, due to images being used that had incorrect orientation metadata in them, probably because of tools that rotated the image data but left the metadata untouched. So it is useful for quickly fixing this. But these were all elements.)

If image-orientation doesn't apply to decorative images, then there is no easy way to opt out of any breakage that's occurred. So I would prefer to have it apply to all content and decorative images that the element has, and then not bother with defining decorative images.

I don't have a strong opinion on cursor and resources linked to with / elements, but I think it would be fine not to them apply there. It's unlikely people are using JPEGs with orientation data in them.

My view of orientation metadata is that it's an implementation detail of the image, and the only reason that we are allowing control of whether it is honored is to help authors manage this change in behavior.

[faceless2]

I agree with Cam on all his points. If EXIF applies to all images, then I think image-orientation has to as well.

[css-meeting-bot]

The CSS Working Group just discussed [css-images-3] clarify which images image-orientation applies to, and agreed to the following:

• RESOLVED: Make from-image's none value apply to background and any other images

The full IRC log of that discussion

Topic: [css-images-3] clarify which images image-orientation applies to
github: https://github.com//issues/5245
heycam: We have image orientation property that allows us turn off auto-re-orientation.
heycam: Some discussion about widening to all images a few months ago. Resolution from there was orientation meta data should be honored for decorative images. Makes sense.
heycam: Wasn't discussed if properties should be extended. As MIke noticed we have contradictory WPTs. Should image-orientation apply to decorative images?
q+
heycam: My feeling is it should b/c real purpose of property is let authors opt-out and it doesn't make sense to limit that to a subset of images
ack dino
dino: Question is what would you do if want decorative to do one thing and non-decorative do another
heycam: You would need to introduce some other way of override orientation, maybe with image value syntax, or correct images
dino: Sort of leading question. In many cases will have to fix image or change content. Is it worth having this apply everywhere? I don't know
heycam: From experience of bug reports they were all content images. jpegs with image data rotated but tool didn't update meta data so new image-orientation made them wrong. Didn't get cases on decorative
fantasai: We had this discussion before and I believe resolution was only to apply to content. Idea was if we needed to apply to other we would introduce image notation syntax. This discussion was before auto xif but that was original discussion and conclusion
Rossen_: Is that 4165?
fantasai: older
fantasai: Let me change DoC
s/change/see if it’s in the/
s/xif/EXIF
https://lists.w3.org/Archives/Public/www-style/2012Mar/0412.html
Rossen_: Where does this leave the current issue?
fantasai: Here's the issue. #50 in DoC. Conclusion was repalced elemtns and generated content but not background image
fantasai: Or any other images in CSS
fantasai: Images which are considered part of content are effected but no other
faceless2_: If purpose is undo exif makes sense to apply, but it's not weird to not. As long as it's documented.
"It applies only to content images (e.g. replaced elements and generated content), not decorative images (such as background-image)."
fantasai: Spec says applies to content not decorative.
https://drafts.csswg.org/css-images-3/#the-image-orientation
fantasai: Anything that's a replaced element in css model it applies to that. Everything else is not effected.
q+
ack fantasai
fantasai, you wanted to discuss history
heycam: Seems like notion of this property has changed over time. Original I want to set an orientation on an image. Now proprty is forI I have a problem caused by change in browsers and this gets me the old way back so I don't have to fix images I'm serving. In that PoV makes sense to apply as broadly as possible. Let's author set one property which goes to all images
This was issue 50 in the 2012 LC Disposition of Comments https://drafts.csswg.org/css-images-3/issues-lc-2012
chrishtr: I agree with heycam reasoning and that's what Chrome impl. There were cases where it was useful behavior for site compat
webkit agrees with cameron too
fantasai: If we make this change it should only affect none value and others shouldn't apply to decorative
ack chrishtr
heycam: I think we have resolution to remove other values.
fantasai: It's deprecated but has been impl and shipped in multiple impl, jsut nto browsers. We can remove from next spec level, but there needs to be a spec with those.
chrishtr: Who impl?
fantasai: Some printer based technology sihpped with onboard layout in printer chipset.
chrishtr: Okay so we say rotate ones won't apply to style images
fantasai: And support for that is marked as optional so browsers don't need to impl. But there needs to be spec for it
Rossen_: Any changes to L3 images?
fantasai: Yes if we want to make from-image and none value apply to background and other images we need to change spec to say it's all images associated with element
Rossen_: Is this going to be still valid for printer or are they okay b/c L2?
fantasai: They don't impl from-image I believe
Rossen_: Sorry, confused by statement that there were impl
fantasai: If we make change for other values than yes. THat's why I'm saying it shouldn't apply to the other values
Rossen_: Would that solution work for everybody
heycam: sgtm
Rossen_: Prop: Make from-image and none value apply to background and any other images?
fantasai: angle only applies to content
Rossen_: none value on from-image
Rossen_: Additional comments?
smfr: Can we resolve on cursor behavior and linked meta that are in GH issue?
Rossen_: Would this be the exception on cursor?
smfr: Makes sense for cursor to have same. Link and meta I don't know since usually don't apply CSS to them
fantasai: Not sure what a link or meta element would do
heycam: Some cases about favicons and similar images
smfr: First comment in issues has list of interesting items like embed object. We need to be explicit which of these we include
chrishtr: Canvas is an imparative API
smfr: Not sure abotu canvas I think right is API to expose exif to JS or extend draw image API
presumably you can put images on link or meta via background-image or ::before and content property etc.
fantasai: Rest it should apply, every image associated to the element is effected as far as CSS is concerned. If it's a replaced element it's applied. Decorative elements would include background iamge and cursor
smfr: Source graphic in SVG
fantasai: Replaced element, isn't it?
smfr: I think it's paint source or whatveer it's called
heycam: I think that's rendered content of some subelement on dom tree
fantasai: Do you do image orientation of source, using, or neither element
heycam: Similar to canvas-draw-image b/c can reference another image element. Have minor incompat on where to look already. I think those cases can be handled in specs that define what it's referencing
smfr: Agree. We can resolve on what we discussed and say SVG may need refining
heycam: I'm happy to make cursor effected
Rossen_: Should we try to resolve on this and heycam or someone can open an issue on SVG to makes sure there's no additional items to associate to this decision?
smfr: sgtm
s/smfr/heycam/
Rossen_: Still previous resolution. Objections?
RESOLVED: Make from-image's none value apply to background and any other images

[fantasai]

Because the spec was pretty clear that it does not apply to background images etc., only to replaced elements (including generated content).