-
Notifications
You must be signed in to change notification settings - Fork 719
[css-highlight-api] Figure out how highlights are exposed to the accessibility tree #6498
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
To sighted users, the highlights are only visible through the way they are styled. There is no other way to know what they're for. Maybe the same should be true of the highlights? Getting the highlight itself described through some text sounds like it might be more convenient, but I'm not sure where we'd pull that description from, how to deal with multiple languages, or the potential discrepancies between the descriptive text and what the highlight is actually used for. |
And whatever gets decided here should likely match what is decided for #6417 |
How are things like these exposed to the accessibility tree today?
We don't necessarily have to do the same, but it'd be good to start with an understanding of how these are handled. |
#4601 seems to have some related remarks. |
UserAgent detected spelling and grammar errors (i.e. ::spelling-error and ::grammar-error) in Chromium do have a mapping to various AT APIs. These are serialized as part of the accessibility tree and as far as I can tell, this is not part of any of the Aria-related standards. For example on UIA on Windows spelling errors are exposed as a text range with an AnnotationType_SpellingError. Not sure how this would fit into the spec, but perhaps is a path forward (following the UIA example, there is an AnnotationType_Highlight). For completeness, spelling and grammar errors that are computed by the webpage can be annotated with aria-invalid. The corresponding concept for highlights would probably be the mark role (see https://w3c.github.io/aria/). However, these require additional DOM nodes, which is counter to the purpose of the HighlightAPI. |
I agree with @dlibby- that for the question of how to expose the highlight ranges to ATs we should be able to follow the lead of how ::spelling-error and ::grammar-error are exposed in Chromium, at least in part. For expressing the semantics of the highlight ranges, the most flexible option seems to be to add a An alternative I'd like to consider is adding an enum to enum HighlightType {
"highlight",
"spellingerror", // Exposed to ATs in the same way as ::spelling-error
"grammarerror", // Exposed to ATs in the same way as ::grammar-error
"textmatch", // e.g. for custom find-on-page results
// ...and any others for the common use cases we can think of.
};
[Exposed=Window]
interface Highlight {
constructor(AbstractRange... initialRanges);
setlike The benefit of this is that we're not relying on author-specified strings, so it's easier for developers and it solves the issue of localization. An enumerated set of types could also be easier to expose to ATs. The disadvantage is that use cases we don't anticipate won't have any good choices so authors will have to fall back to the generic "highlight". I don't think we'd be locked into the initial set of HighlightTypes though; we could probably add more if additional common uses cases arise. |
Tagging for a11y-tracker, since the dup that was closed was so tagged. |
Another thing I'll add here is that an advantage of an enum over a string is that when OSs have some first-class representation for any of the use cases in the enum, user agents could plug into them directly. For example, custom spell check implementations using Highlight API could be represented to accessibility tools in the exact same way as ranges marked by the native spell-checker. In UIA this might AnnotationType_SpellingError, or NSAccessibilityMarkedMisspelledTextAttribute on Mac. This would be a more consistent experience for AT users and would save the author the trouble of localizing a "spelling error" string. It's also feasible that in the future ATs could become more powerful and offer first-class representations of more of these use cases, and it would be great if authors can provide metadata that provides a better experience in those cases. If needed, in an L2 of the spec a string attribute could be added to |
There's been some delay on agenda space opening up to discuss this during CSSWG calls, so I wanted try again to see if we could make some progress here in the issue. Considering the enum approach considered here and looking more at the current capabilities of UIA's Annotation Type Identifiers and Mac's Attributed String Keys, I'd like to propose starting with an enum with just these: enum HighlightType {
// Exposed with AnnotationType_Highlight for UIA, accessibilityCustomText for Mac
"highlight",
// Exposed with AnnotationType_SpellingError for UIA, NSAccessibilityMisspelledTextAttribute for Mac
"spellingerror",
// Exposed with AnnotationType_GrammarError for UIA, NSAccessibilityMisspelledTextAttribute for Mac
"grammarerror"
// More can be added as use cases emerge and accessibility platforms add more text semantic types
};
[Exposed=Window]
interface Highlight {
constructor(AbstractRange... initialRanges);
setlike<AbstractRange>;
attribute long priority;
attribute HighlightType type; // Exposed to ATs. Default is "highlight".
}; This approach lets custom spellcheck achieve the same experience with accessibility tools as native spellcheck for UIA and Mac today, and leaves room to add more entries to the enum as use cases an support emerge. A custom string could be added in an L2 if the need for that level of flexibility emerges. @frivoal @sanketj @dlibby- @lilles, any thoughts on this approach? |
The CSS Working Group just discussed
The full IRC log of that discussion |
As resolved in w3c/csswg-drafts#6498, this CL adds a new attribute to Highlight which will be used for a more precise Accessibility interpretation of the highlights. (Exposure to ATs to be done in a following up CL) Change-Id: I7a84a11e5f52b5b950d807ceb276b74395b1b164
As resolved in w3c/csswg-drafts#6498, this CL adds a new attribute to Highlight which will be used for a more precise Accessibility interpretation of the highlights. (Exposure to ATs to be done in a following up CL) Change-Id: I7a84a11e5f52b5b950d807ceb276b74395b1b164
As resolved in w3c/csswg-drafts#6498, this CL adds a new attribute to Highlight which will be used for a more precise Accessibility interpretation of the highlights. (Exposure to ATs to be done in a following up CL) Change-Id: I7a84a11e5f52b5b950d807ceb276b74395b1b164 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3214997 Commit-Queue: Fernando FioriReviewed-by: Dan Clark Reviewed-by: Rune Lillesveen Cr-Commit-Position: refs/heads/main@{#930852}
As resolved in w3c/csswg-drafts#6498, this CL adds a new attribute to Highlight which will be used for a more precise Accessibility interpretation of the highlights. (Exposure to ATs to be done in a following up CL) Change-Id: I7a84a11e5f52b5b950d807ceb276b74395b1b164 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3214997 Commit-Queue: Fernando FioriReviewed-by: Dan Clark Reviewed-by: Rune Lillesveen Cr-Commit-Position: refs/heads/main@{#930852}
As resolved in w3c/csswg-drafts#6498, this CL adds a new attribute to Highlight which will be used for a more precise Accessibility interpretation of the highlights. (Exposure to ATs to be done in a following up CL) Change-Id: I7a84a11e5f52b5b950d807ceb276b74395b1b164 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3214997 Commit-Queue: Fernando FioriReviewed-by: Dan Clark Reviewed-by: Rune Lillesveen Cr-Commit-Position: refs/heads/main@{#930852}
…ighlight, a=testonly Automatic update from web-platform-tests Highlight API: Add 'type' attribute to Highlight As resolved in w3c/csswg-drafts#6498, this CL adds a new attribute to Highlight which will be used for a more precise Accessibility interpretation of the highlights. (Exposure to ATs to be done in a following up CL) Change-Id: I7a84a11e5f52b5b950d807ceb276b74395b1b164 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3214997 Commit-Queue: Fernando FioriReviewed-by: Dan Clark Reviewed-by: Rune Lillesveen Cr-Commit-Position: refs/heads/main@{#930852} -- wpt-commits: 2274db5eb189b8fb4d8e0efec4dcf5c9d03e8a0a wpt-pr: 31172
…ighlight, a=testonly Automatic update from web-platform-tests Highlight API: Add 'type' attribute to Highlight As resolved in w3c/csswg-drafts#6498, this CL adds a new attribute to Highlight which will be used for a more precise Accessibility interpretation of the highlights. (Exposure to ATs to be done in a following up CL) Change-Id: I7a84a11e5f52b5b950d807ceb276b74395b1b164 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3214997 Commit-Queue: Fernando FioriReviewed-by: Dan Clark Reviewed-by: Rune Lillesveen Cr-Commit-Position: refs/heads/main@{#930852} -- wpt-commits: 2274db5eb189b8fb4d8e0efec4dcf5c9d03e8a0a wpt-pr: 31172
At the TPAC session, it was mentioned that arbitrary strings would have disadvantages. In lieu of this, I wonder if the generic highlight item in the enum could have a secondary field for color? In education, often a variety of highlight colors are made available and individual instructors and groups use those different colors for different meanings. For example, an instructor might tell a student they need to clarify the sections highlighted in yellow. If the color could be exposed with the information that the content is highlighted, this would allow AT users to participate in these informal meaning assignments. |
Minutes from the CSS & APA joint meeting at TPAC: https://www.w3.org/2021/10/20-cssa11y-minutes.html#t02 |
@SuzanneTaylor If the highlight range is being used to change the text color or background color of a range of text, the author will have had to include a rule for that in the |
For the use case of highlights in application-specific highlighting. Using this CSS API can an author define multiple different highlights and provide the associated text meaning of a particular color of highlight that can be relayed to AT using the CSS attributes? As an example, an app that scans through large amounts of text to highlight certain aspects particular to the search (e.g. to point out sentiments or emotions in text, such as positive, negative, neutral, happy, angry, etc.) |
That makes sense, but how will colors specified as numbers become human-friendly words? With contrast requirements already restricting color choices for the highlights, background and text colors, I don't think being limited to the named colors will work well. |
We’d explored adding a string property to
It should be possible for accessibility tools to turn colors specified with numbers back into human-friendly names. I confirmed that Windows Narrator does this for any random RGB values I try. Here's a small example where the colors are randomly typed RGB values: https://codepen.io/daniec/pen/xxLrqqo |
love it! (assuming, of course, highlights and their human-readable colors will be expected to be announced at default verbosity levels) |
Would it be possible also to indicate in any documentation about browser accessibility support for this feature, that browsers should consider a setting (or maybe there should be a media query for this) that users could select to have the name of the color appear visually? You would not want ALL CSS colors to appear visually of course, just those associated with highlights. One possible visual layout could be smaller text above the highlighted text, overlapping onto the highlight a bit. |
I think it's a good idea to at least add a note in the spec that browsers should ensure that the colors of the highlights are made available to accessibility. I'm not yet sure whether there's a need for a setting to expose the highlight colors in a special way. In many (most?) cases, the specific color of the highlights won't matter semantically, e.g. with custom find-on-page, spelling/grammar error, and highlights used for generic emphasis of text. For the rarer cases where color matters, users would know from context on the page that the colors are meaningful and would be able to adjust their accessibility tool's settings accordingly to get that info. That said I'd like to continue exploring an option like this in the L2 of the spec when we've got more experience with how the API ends up being used in the wild. |
As resolved in w3c/csswg-drafts#6498, this CL adds a new attribute to Highlight which will be used for a more precise Accessibility interpretation of the highlights. (Exposure to ATs to be done in a following up CL) Change-Id: I7a84a11e5f52b5b950d807ceb276b74395b1b164 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3214997 Commit-Queue: Fernando FioriReviewed-by: Dan Clark Reviewed-by: Rune Lillesveen Cr-Commit-Position: refs/heads/main@{#930852}
…6724) * Add `type` attribute to `Highlight` * Say what UAs should do with the type. Explain why initial set of types was chosen.
@atanassov recently reminded us that figure out how highlights are exposed to the accessibility tree. This had been know for a while, but we didn't have an issue I could find, so here's one.
The text was updated successfully, but these errors were encountered: