Spatial Navigation

[frivoal]

Hello TAG！

I'm requesting a TAG review of:

• Name: Spatial Navigation
• Specification URL: https://wicg.github.io/spatial-navigation/
• Explainer, Requirements Doc, or Example code:
  • explainer
  • Overview section in the spec
  • Several examples in the spec, including sample code in the api section
  • Polyfill (this is early stage, has bugs, and is not fully spec compliant yet. We do not request a review of the polyfill, but it may be useful to try things out).
  • Demo page using the polyfill
• Tests: just starting, there a few tests in our wicg repo. Will submit to WPT later.
• Primary contacts: @frivoal & @jihyerish

Further details (optional):

• Relevant time constraints or deadlines: No strict deadline, but speedy feedback appreciated.

• I have read but not yet filled out the Self-Review Questionnare on Security and Privacy.

  Known issues:

  • Review the security model of how we interact with iframes WICG/spatial-navigation#28
  • [css-nav-1] hostile iframes WICG/spatial-navigation#58

• I have reviewed the TAG's API Design Principles

  • Open issue about 3.1 add event handler attributes WICG/spatial-navigation#60

  • Not following advice from 3.2, in favor of this sentence from the DOM spec:

    Apart from signaling, events are sometimes also used to let an application control what happens next in an operation. For instance as part of form submission an event whose type attribute value is "submit" is dispatched. If this event’s preventDefault() method is invoked, form submission will be terminated.

    Can consider alternative approaches, but we think the current design makes sense in this case. If you disagree, please give an example of what would be better (including how it would work with the examples in the spec).

You should also know that...

The spec isn't complete yet, so this is more of an opportunity to review the proposed API.
Unless someone raises a big issue, high level design is complete. Polyfill implementation started. Details still need more work, and native implementation is not started yet (but we will start lobbying for that soon). In other words, big changes are still possible at this stage, but it will soon start getting more expensive.

We'd prefer the TAG provide feedback as (please select one):

• open issues in our Github repo for each point of feedback
• open a single issue in our Github repo for the entire review
• leave review feedback as a comment in this issue and @-notify [github usernames]

[cynthia]

Hey all,

Thanks for filing a review. We noted excitement about this at TPAC and are glad to have the opportunity to work with you on this. Thanks also for the detailed explainer.

We spoke with folks working on this recently in an ad-hoc discussion, wanted to get some of these thoughts written down.

Hostile iframes seem like a serious issue, but we're curious about how they're different to other sorts of focus capturing issues; e.g. the fullscreen API. Those APIs preserve a keyboard shortcut to allow escape. Is something like thpossible? The proposed feature policy seems like a promising direction.

The traversal (distance computation) algorithm seems to have properties which result in unexpected (at least from a user's perspective). A potential improvement has been noted in your issue tracker - WICG/spatial-navigation#122 (comment) although there are probably better ways to approach this.

Are Focusable Areas seem to have a correspondence with Intersection Observers. Has a similar set of rootMargin and threshold options been considered?

An interesting point of feedback from other communities was around the right-left (start on one element, traversal, where focus does not return to the element the user came from, this seems like a limitation that should either be addressed or explicitly noted.

Minor nit: The spec text for defining P1 and P2 could be clearer.

[jihyerish]

Thanks for the detailed feedback!

I think some of the issues can be solved as below:

Hostile iframe

Yes, I think the Feature policy can solve this issue.
We are discussing this in this approach, and I've written the proposal about the Feature Policy:

• [css-nav-1] hostile iframes WICG/spatial-navigation#58 (comment)
  (More detail: Proposal: navigation-override w3c/webappsec-permissions-policy#175)

Distance computation algorithm

1. Define more reasonable distance function

   I've started to investigate the distance measuring algorithm.
   Using the listed test cases, we can choose a reasonable algorithm.
   The test cases are written with the template of the web platform test.
   Please note that the result of each case doesn't match with the current spec, but the UX point of view decides it.

   • Test page: https://jihyerish.github.io/spatial-navigation/tests/ux/result.html

   Also, the additional distance algorithms will be added.

2. Define P1 from the starting point and P2 from the candidate element for the distance function

   Depending on the result of "Defining the distance function", we can also decide how to define P1 and P2 for the distance function which is described in the current spec.

   • Related issue: [css-nav-1] Define points precisely for measuring the distance  w3c/csswg-drafts#3353

Focusable Areas

I'm not sure I understand this point clear.
Is this (A) or (B)?
(A) Add the options object similar to the IntersectionObserver() for the focusableAreas()
(B) Use the IntersectionObserver instead of the focusableAreas()

If it is (A), using the root, threshold field of the options object seems useful.
But is it possible to apply this behavior to the iframe which is the cross-origin from the current document?

(B) doesn't seem to match with the intention of the focusableAreas().
The target elements and the expected result for both API are different.
The focusableAreas() returns the visible focusable element inside the specific element (in general, container element).
However, the IntersectionObserver let us know whether the specified element is visible or not inside the container element.

Returnable focus traversal

(e.g. pressing the right-left-right arrow keys makes the focus return to the initial element)
This behavior is useful, but it seems hard to be handled by combining with the distance measuring algorithm.
I can think of it the other way around, adding the option for taking the already visited element higher priority in the selecting the best candidate step, after measuring the distance.

[jihyerish]

Hostile iframe
Yes, I think the Feature policy can solve this issue.
We are discussing this in this approach, and I've written the proposal about the Feature Policy:

WICG/spatial-navigation#58 (comment)
(More detail: w3c/webappsec-permissions-policy#175)

This had been solved with using feature policy and the spec had been updated:

• https://drafts.csswg.org/css-nav-1/#policy-feature

[jihyerish]

Distance computation algorithm

I've made progress on improving the distance computation algorithm.

1. Define more reasonable distance function

• Related issue: [css-nav-1] Improve the distance function w3c/csswg-drafts#3384

2. Define P1 from the starting point and P2 from the candidate element for the distance function

• Related issue: [css-nav-1] Define points precisely for measuring the distance  w3c/csswg-drafts#3353

Please see the overall proposal for changing the distance computation algorithm in here.

[dbaron]

@cynthia and I are looking at this in a breakout at the TAG's Reykjavík face-to-face meeting.

One concern that I have is the bit at the very end of the explainer about spatial navigation and HTMLFormElement, which links to a separate document. I have a few concerns here:

• form controls aren't necessarily implemented using shadow DOM, but might be in the future. However, there probably should be text saying both what happens for form controls and for shadow DOM.
• whether it's using shadow DOM or not, it's important not to break encapsulation. This means that the insides of the form control or shadow tree can't be visible to the outside. Yet it still should be possible to navigate within the parts of the shadow tree or form control if there are multiple parts. So it seems like perhaps what spatial navigation exposes inside of a shadow tree should probably be different from what it exposes outside a shadow tree. That is, perhaps there should be a way for the shadow tree to "capture" some of the spatial navigations to be movement inside of it, and then in other cases allow the navigations to move outside of it. But it's probably worth running this past folks involved in w3c/webcomponents.
• the guidelines on dealing with form controls seem a bit too prescriptive -- they seem to assume particular designs for form controls that may not be the case

(Note that these comments are based on the explainer; I haven't looked at the spec.)

@cynthia will add some comments on other topics that we looked at.

[cynthia]

Apologies for the delayed feedback! @dbaron and I looked at the incorporated changes and discussed this during the Iceland F2F. Some points of feedback:

Completely turning off the feature with feature policy as a mitigation towards hostile iframes seems a bit extreme. Since the actual surface of attack from hostile iframes is rather narrow, prevention against trapping seems like a more sensible way forward. This is because you would still want to let the users focus into third party iframes, but would want to mitigate against focus trap attacks. Specifically, ad providers would probably not be happy if they are not clickable.

Following that thought, the feature policy name should probably change when this is applied to reflect better exactly what is being allowed/disallowed.

Small question:
What does "absolute distance" mean? I'm suspecting it's an alias to manhattan distance, but that part is rather unclear. (Especially confusing since euclidean distance is always absolute in terms of value since we aren't doing complex space geometry.)

[jihyerish]

Thank you for the feedback!

@dbaron
I agree with your concern. Considering shadow DOM and form control is the hardest part of spatial navigation. Nonetheless, the spec needs to describe how to handle those.

So it seems like perhaps what spatial navigation exposes inside of a shadow tree should probably be different from what it exposes outside a shadow tree. That is, perhaps there should be a way for the shadow tree to "capture" some of the spatial navigations to be movement inside of it, and then in other cases allow the navigations to move outside of it.

I have a vague thought which is making the navigation event do not cross shadow DOM boundaries.
However, I need to think deeply about this and talk with folks working in webcomponents

There is another approach to solve this issue, and the discussion about it is still going on.
If there is any progress about it, I'll share it here.

@cynthia

Completely turning off the feature with feature policy as a mitigation towards hostile iframes seems a bit extreme.

navigation-override feature policy doesn't block the entire spatial navigation feature.
It just restricts iframe from using Navigation event.

There is navnotarget event which occurs when there isn't any element which will get the focus next within the current container. (iframe is also considered as a container.)
In general, after firing navnotarget , the focus moves out of the container.

That means, if the hostile iframe prevents default of navnotarget , the focus is trapped inside it.

To avoid this kind of attempt, navigation-override makes the hostile iframe cannot use Navigation event.

Also, there is an on-going discussion about the new feature policy focus-without-user-activation.
(related github issue: w3c/webappsec-permissions-policy#273)
I think navigation-override needs to be considered with this feature.
(related github issue: w3c/csswg-drafts#3656)

What does "absolute distance" mean? I'm suspecting it's an alias to manhattan distance, but that part is rather unclear.

Oh, the expression is ambiguous. It means that the distance considering only one axis.
If the navigation direction is right or left, it measures distance only along the x-axis.

[cynthia]

Thank you for the clarification. We discussed this during today's teleconference and as for the state of the spec as of today, we believe we have provided all the technical feedback we felt was necessary.

As for the shadow DOM bits and feature policy overlap with another proposal, we'd like to see how the details get fleshed out - and if if feels like there have been substantial enough design changes we'd be happy to look into this again at that point. (Shadow DOM definitely feels like a large enough patch to warrant a second round of reviews.)

We are happy to see this move forward, and thank you for bringing this up with us.

[jihyerish]

Thank you for the review and give us feedback about Spatial Navigation!

Recently, another design proposal about the spec is under consideration.
This proposal is also related to shadow DOM and feature policy, but it needs more experiment and brainstorming.
If the experiment is mature enough to be described in the spec, could I request the review again?

Thank you again for giving valuable feedback about this!

[cynthia]

If the experiment is mature enough to be described in the spec, could I request the review again?

It doesn’t necessarily have to be a spec format, especially if multiple proposals are on the table. Something like a simple explainer with example code/demo works. We also have a dispute resolution issue template in the event you prefer that. Otherwise feel free to reppen when you have something you’d like us to look at.