From 78c665d9da49092c33907842857a5df1c6a87984 Mon Sep 17 00:00:00 2001 From: Ada Rose Cannon Date: Mon, 21 Aug 2023 14:01:30 +0100 Subject: [PATCH 1/7] start transient intent addition --- index.bs | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/index.bs b/index.bs index 3be14e2a..476f71b6 100755 --- a/index.bs +++ b/index.bs @@ -1823,7 +1823,8 @@ enum XRHandedness { enum XRTargetRayMode { "gaze", "tracked-pointer", - "screen" + "screen", + "transient-intent" }; [SecureContext, Exposed=Window] @@ -1845,11 +1846,16 @@ The targetRayMode attribute describes t - gaze indicates the target ray will originate at the viewer and follow the direction it is facing. (This is commonly referred to as a "gaze input" device in the context of head-mounted displays.) - tracked-pointer indicates that the target ray originates from either a handheld device or other hand-tracking mechanism and represents that the user is using their hands or the held device for pointing. The orientation of the target ray relative to the tracked object MUST follow platform-specific ergonomics guidelines when available. In the absence of platform-specific guidance, the target ray SHOULD point in the same direction as the user's index finger if it was outstretched. - screen indicates that the input source was an interaction with the canvas element associated with an inline session's output context, such as a mouse click or touch event. + - transient-intent indicates that the input source was generated as part of an operating system interaction intent rather than a specific piece of hardware. Some examples are user intents based on information too sensitive to expose directly such as gaze, synthesised inputs from web driver or inputs generated by assistive technologies. This should only be used for assistive technologies if it is also used as a primary input so as to not inadvertently indicate that assistive technology is being used as per the W3C design principals. The targetRaySpace attribute is an {{XRSpace}} that has a [=native origin=] tracking the position and orientation of the preferred pointing ray of the {{XRInputSource}} (along its `-Z` axis), as defined by the {{targetRayMode}}. +For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{targetRaySpace}} represents the ray to the interaction target at the start of the interaction, the {{targetRaySpace}} should not change for the lifetime of the inputSource. + The gripSpace attribute is an {{XRSpace}} that has a [=native origin=] tracking to the pose that should be used to render virtual objects such that they appear to be held in the user's hand. If the user were to hold a straight rod, this {{XRSpace}} places the [=native origin=] at the centroid of their curled fingers and where the `-Z` axis points along the length of the rod towards their thumb. The `X` axis is perpendicular to the back of the hand being described, with the back of the user's right hand pointing towards `+X` and the back of the user's left hand pointing towards `-X`. The `Y` axis is implied by the relationship between the `X` and `Z` axis, with `+Y` roughly pointing in the direction of the user's arm. +For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{gripSpace}} should be the associated user gesture if there is one, otherwise it should be null. The {{gripSpace}} can change over time as usual. + The {{gripSpace}} MUST be `null` if the input source isn't inherently trackable such as for input sources with a {{targetRayMode}} of {{XRTargetRayMode/"gaze"}} or {{XRTargetRayMode/"screen"}}. The profiles attribute is a [=/list=] of [=input profile name=]s indicating both the prefered visual representation and behavior of the input source. @@ -1941,7 +1947,13 @@ When an [=XR input source=] |source| for {{XRSession}} |session| has its [=prima Transient input {#transient-input} --------------- -Some [=/XR device=]s may support transient input sources, where the [=XR input source=] is only meaningful while performing a transient action, either the [=primary action=] for a [=primary input source=], or a device-specific auxiliary action for an [=auxiliary input source=]. An example would be mouse, touch, or stylus input against an {{XRSessionMode/"inline"}} {{XRSession}}, which MUST produce a transient {{XRInputSource}} with a {{targetRayMode}} set to {{screen}}, treated as a [=primary action=] for the [=primary pointer=], and as a non-primary [=auxiliary action=] for a non-primary pointer. [=Transient input sources=] are only present in the session's [=list of active XR input sources=] for the duration of the [=transient action=]. +Some [=/XR device=]s may support transient input sources, where the [=XR input source=] is only meaningful while performing a transient action, either the [=primary action=] for a [=primary input source=], or a device-specific auxiliary action for an [=auxiliary input source=]. + +One example would be mouse, touch, or stylus input against an {{XRSessionMode/"inline"}} {{XRSession}}, which MUST produce a transient {{XRInputSource}} with a {{targetRayMode}} set to {{screen}}, treated as a [=primary action=] for the [=primary pointer=], and as a non-primary [=auxiliary action=] for a non-primary pointer. + +Another example would be intents from the operating system with input derived from sensitive information that cannot be exposed directly, such as interactions based on gaze. These produce a transient {{XRInputSource}} with a {{targetRayMode}} set to {{transient-intent}}, treated as a [=primary action=]. + +[=Transient input sources=] are only present in the session's [=list of active XR input sources=] for the duration of the [=transient action=]. [=Transient input sources=] follow the following sequence when handling [=transient actions=] instead of the algorithms for non-transient [=primary actions=]: From 5347f1dea602e6452380567b67fb0c02b3883384 Mon Sep 17 00:00:00 2001 From: Brandon Jones Date: Wed, 21 Jun 2023 08:57:49 -0700 Subject: [PATCH 2/7] Clarify getPose behavior with visibile-blurred The "poses may be reported" algorithm previously indicated that poses could only be reported if the visibility state was "visible", but that appears to be a mistake originating from the fact that "visible-blurred" was added later. The description of "visible-blurred" mentioned throttled head poses, so obviously the intention was to allow some poses in that state, just not input poses. This change alters the "poses may be reported" algorithm to only suppress poses when the visiblity state is "hidden", and adds an explicit step to the "populate the pose" algorithm that prevents input poses from being returned if the visibility state is "visible-blurred" --- index.bs | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/index.bs b/index.bs index 476f71b6..42a9b924 100755 --- a/index.bs +++ b/index.bs @@ -1259,6 +1259,7 @@ To populate the pose of an {{XRSpace}} |space| in an {{XRSpace}} |bas 1. If |space|'s [=XRSpace/session=] does not equal |session|, throw an {{InvalidStateError}} and abort these steps. 1. If |baseSpace|'s [=XRSpace/session=] does not equal |session|, throw an {{InvalidStateError}} and abort these steps. 1. Check if [=poses may be reported=] and, if not, throw a {{SecurityError}} and abort these steps. + 1. If |session|'s {{XRSession/visibilityState}} is {{XRVisibilityState/"visible-blurred"}} and |space| or |baseSpace| is associated with an {{XRInputSource}}, set |pose| to `null` and abort these steps. 1. Let |limit| be the result of whether [=poses must be limited=] between |space| and |baseSpace|. 1. Let |transform| be |pose|'s {{XRPose/transform}}. 1. Query the [=/XR device=]'s tracking system for |space|'s pose relative to |baseSpace| at the |frame|'s [=XRFrame/time=], then perform the following steps: @@ -2619,7 +2620,7 @@ When based on sensor data, {{XRPose}} and {{XRViewerPose}} will expose [=sensiti To determine if poses may be reported to an {{XRSession}} |session|, the user agent MUST run the following steps: 1. If |session|'s [=relevant global object=] is not the [=current global object=], return `false`. - 1. If |session|'s {{XRSession/visibilityState}} is not {{XRVisibilityState/"visible"}}, return `false`. + 1. If |session|'s {{XRSession/visibilityState}} is {{XRVisibilityState/"hidden"}}, return `false`. 1. Determine if the pose data can be returned as follows:
: If the pose data is known by the user agent to not expose fingerprintable sensor data From cba3dc9c895fdf8a94a7dbc92aeca0c50d222729 Mon Sep 17 00:00:00 2001 From: Ada Rose Cannon Date: Wed, 30 Aug 2023 14:52:00 +0100 Subject: [PATCH 3/7] add allowance for system ui for safety --- index.bs | 2 ++ 1 file changed, 2 insertions(+) diff --git a/index.bs b/index.bs index 42a9b924..75f2ae5a 100755 --- a/index.bs +++ b/index.bs @@ -495,6 +495,8 @@ In this document, the term inline session is synonymous with an {{inl [=Immersive sessions=] MUST provide some level of [=viewer=] tracking, and content MUST be shown at the proper scale relative to the user and/or the surrounding environment. Additionally, [=Immersive sessions=] MUST be given exclusive access to the [=immersive XR device=], meaning that while the [=immersive session=] is {{XRVisibilityState/"visible"}} the HTML document is not shown on the [=immersive XR device=]'s display, nor does content from any other source have exclusive access. [=Exclusive access=] does not prevent the user agent from overlaying its own UI, however this UI SHOULD be minimal. +Note: UA may choose to overlay content for accessibility or safety such as guardian boundaries, obstructions or the user's hands when there are no alternative input sources. + Note: Future specifications or modules may expand the definition of [=immersive session=] to include additional session modes. Note: Examples of ways [=exclusive access=] may be presented include stereo content displayed on a virtual reality headset. From 501dbc8ddfe0342fe5be17468cdd54a57b310596 Mon Sep 17 00:00:00 2001 From: Brandon Jones Date: Fri, 1 Sep 2023 11:12:22 -0700 Subject: [PATCH 4/7] Fix Bikeshed build --- index.bs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/index.bs b/index.bs index 75f2ae5a..94440cfb 100755 --- a/index.bs +++ b/index.bs @@ -1737,13 +1737,13 @@ To obtain the matrix for a given {{XRRigidTran 1. If the operation {{IsDetachedBuffer}} on [=XRRigidTransform/internal matrix=] is `false`, return |transform|'s [=XRRigidTransform/internal matrix=]. 1. Let |translation| be a new [=matrix=] which is a column-vector translation matrix corresponding to {{XRRigidTransform/position}}. Mathematically, if {{XRRigidTransform/position}} is `(x, y, z)`, this matrix is - Mathematical expression for column-vector translation matrix + Mathematical expression for column-vector translation matrix 1. Let |rotation| be a new [=matrix=] which is a column-vector rotation matrix corresponding to {{XRRigidTransform/orientation}}. Mathematically, if {{XRRigidTransform/orientation}} is the unit quaternion (qx, qy, qz, qw), this matrix is - Mathematical expression for column-vector rotation matrix + Mathematical expression for column-vector rotation matrix 1. Set |transform|'s [=XRRigidTransform/internal matrix=] to a [=new=] {{Float32Array}} in the [=relevant realm=] of |transform| set to the result of multiplying |translation| and |rotation| with |translation| on the left (`translation * rotation`) in the [=relevant realm=] of |transform|. Mathematically, this matrix is - Mathematical expression for matrix of multiplying translation and rotation with translation on the left + Mathematical expression for matrix of multiplying translation and rotation with translation on the left 1. Return |transform|'s [=XRRigidTransform/internal matrix=]. From 6010a8fbf42ad4f6048401aedb9ce5cac0c9ada0 Mon Sep 17 00:00:00 2001 From: Ada Rose Cannon Date: Mon, 25 Sep 2023 16:22:48 +0100 Subject: [PATCH 5/7] pose update first draft --- index.bs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.bs b/index.bs index 94440cfb..72531ffc 100755 --- a/index.bs +++ b/index.bs @@ -1853,7 +1853,7 @@ The targetRayMode attribute describes t The targetRaySpace attribute is an {{XRSpace}} that has a [=native origin=] tracking the position and orientation of the preferred pointing ray of the {{XRInputSource}} (along its `-Z` axis), as defined by the {{targetRayMode}}. -For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{targetRaySpace}} represents the ray to the interaction target at the start of the interaction, the {{targetRaySpace}} should not change for the lifetime of the inputSource. +For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{targetRaySpace}} represents the ray to the interaction target at the start of the interaction. The pose for this should be static within another space the user controls. For example, the {{gripSpace}} of the same input source, a space on another input source or the viewer space. This allows for the static ray to still be used to still be used for more complex interactions. The gripSpace attribute is an {{XRSpace}} that has a [=native origin=] tracking to the pose that should be used to render virtual objects such that they appear to be held in the user's hand. If the user were to hold a straight rod, this {{XRSpace}} places the [=native origin=] at the centroid of their curled fingers and where the `-Z` axis points along the length of the rod towards their thumb. The `X` axis is perpendicular to the back of the hand being described, with the back of the user's right hand pointing towards `+X` and the back of the user's left hand pointing towards `-X`. The `Y` axis is implied by the relationship between the `X` and `Z` axis, with `+Y` roughly pointing in the direction of the user's arm. From f5792ec132ef9f85c8a920d1277f743fa14c863e Mon Sep 17 00:00:00 2001 From: Ada Rose Cannon Date: Fri, 29 Sep 2023 16:53:56 +0100 Subject: [PATCH 6/7] Gripspace tweak --- index.bs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/index.bs b/index.bs index 72531ffc..c0ff8201 100755 --- a/index.bs +++ b/index.bs @@ -1853,11 +1853,11 @@ The targetRayMode attribute describes t The targetRaySpace attribute is an {{XRSpace}} that has a [=native origin=] tracking the position and orientation of the preferred pointing ray of the {{XRInputSource}} (along its `-Z` axis), as defined by the {{targetRayMode}}. -For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{targetRaySpace}} represents the ray to the interaction target at the start of the interaction. The pose for this should be static within another space the user controls. For example, the {{gripSpace}} of the same input source, a space on another input source or the viewer space. This allows for the static ray to still be used to still be used for more complex interactions. +For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{targetRaySpace}} represents the ray to the interaction target at the start of the interaction. The pose for this should be static within the gripSpace for this XRInput. The gripSpace attribute is an {{XRSpace}} that has a [=native origin=] tracking to the pose that should be used to render virtual objects such that they appear to be held in the user's hand. If the user were to hold a straight rod, this {{XRSpace}} places the [=native origin=] at the centroid of their curled fingers and where the `-Z` axis points along the length of the rod towards their thumb. The `X` axis is perpendicular to the back of the hand being described, with the back of the user's right hand pointing towards `+X` and the back of the user's left hand pointing towards `-X`. The `Y` axis is implied by the relationship between the `X` and `Z` axis, with `+Y` roughly pointing in the direction of the user's arm. -For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{gripSpace}} should be the associated user gesture if there is one, otherwise it should be null. The {{gripSpace}} can change over time as usual. +For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{gripSpace}} should be the associated user gesture if there is one, otherwise it should be another space the user controls such as the ViewerSpace or the gripSpace or the targetRaySpace of another XRInput. This is to allow user the user to still manipulate the targetRaySpace. The {{gripSpace}} MUST be `null` if the input source isn't inherently trackable such as for input sources with a {{targetRayMode}} of {{XRTargetRayMode/"gaze"}} or {{XRTargetRayMode/"screen"}}. From 9374299bc30b9350ca52d7b4de7e74a71c07fb6f Mon Sep 17 00:00:00 2001 From: Ada Rose Cannon Date: Tue, 3 Oct 2023 22:11:44 +0100 Subject: [PATCH 7/7] change enum --- index.bs | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/index.bs b/index.bs index c0ff8201..9452e149 100755 --- a/index.bs +++ b/index.bs @@ -1827,7 +1827,7 @@ enum XRTargetRayMode { "gaze", "tracked-pointer", "screen", - "transient-intent" + "transient-pointer" }; [SecureContext, Exposed=Window] @@ -1849,15 +1849,15 @@ The targetRayMode attribute describes t - gaze indicates the target ray will originate at the viewer and follow the direction it is facing. (This is commonly referred to as a "gaze input" device in the context of head-mounted displays.) - tracked-pointer indicates that the target ray originates from either a handheld device or other hand-tracking mechanism and represents that the user is using their hands or the held device for pointing. The orientation of the target ray relative to the tracked object MUST follow platform-specific ergonomics guidelines when available. In the absence of platform-specific guidance, the target ray SHOULD point in the same direction as the user's index finger if it was outstretched. - screen indicates that the input source was an interaction with the canvas element associated with an inline session's output context, such as a mouse click or touch event. - - transient-intent indicates that the input source was generated as part of an operating system interaction intent rather than a specific piece of hardware. Some examples are user intents based on information too sensitive to expose directly such as gaze, synthesised inputs from web driver or inputs generated by assistive technologies. This should only be used for assistive technologies if it is also used as a primary input so as to not inadvertently indicate that assistive technology is being used as per the W3C design principals. + - transient-pointer indicates that the input source was generated as part of an operating system interaction intent rather than a specific piece of hardware. Some examples are user intents based on information too sensitive to expose directly such as gaze, synthesised inputs from web driver or inputs generated by assistive technologies. This should only be used for assistive technologies if it is also used as a primary input so as to not inadvertently indicate that assistive technology is being used as per the W3C design principals. The targetRaySpace attribute is an {{XRSpace}} that has a [=native origin=] tracking the position and orientation of the preferred pointing ray of the {{XRInputSource}} (along its `-Z` axis), as defined by the {{targetRayMode}}. -For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{targetRaySpace}} represents the ray to the interaction target at the start of the interaction. The pose for this should be static within the gripSpace for this XRInput. +For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-pointer"}} the {{targetRaySpace}} represents the ray to the interaction target at the start of the interaction. The pose for this should be static within the gripSpace for this XRInput. The gripSpace attribute is an {{XRSpace}} that has a [=native origin=] tracking to the pose that should be used to render virtual objects such that they appear to be held in the user's hand. If the user were to hold a straight rod, this {{XRSpace}} places the [=native origin=] at the centroid of their curled fingers and where the `-Z` axis points along the length of the rod towards their thumb. The `X` axis is perpendicular to the back of the hand being described, with the back of the user's right hand pointing towards `+X` and the back of the user's left hand pointing towards `-X`. The `Y` axis is implied by the relationship between the `X` and `Z` axis, with `+Y` roughly pointing in the direction of the user's arm. -For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-intent"}} the {{gripSpace}} should be the associated user gesture if there is one, otherwise it should be another space the user controls such as the ViewerSpace or the gripSpace or the targetRaySpace of another XRInput. This is to allow user the user to still manipulate the targetRaySpace. +For input sources with a {{targetRayMode}} of {{XRTargetRayMode/"transient-pointer"}} the {{gripSpace}} should be the associated user gesture if there is one, otherwise it should be another space the user controls such as the ViewerSpace or the gripSpace or the targetRaySpace of another XRInput. This is to allow user the user to still manipulate the targetRaySpace. The {{gripSpace}} MUST be `null` if the input source isn't inherently trackable such as for input sources with a {{targetRayMode}} of {{XRTargetRayMode/"gaze"}} or {{XRTargetRayMode/"screen"}}. @@ -1954,7 +1954,7 @@ Some [=/XR device=]s may support transient input sources, where the [ One example would be mouse, touch, or stylus input against an {{XRSessionMode/"inline"}} {{XRSession}}, which MUST produce a transient {{XRInputSource}} with a {{targetRayMode}} set to {{screen}}, treated as a [=primary action=] for the [=primary pointer=], and as a non-primary [=auxiliary action=] for a non-primary pointer. -Another example would be intents from the operating system with input derived from sensitive information that cannot be exposed directly, such as interactions based on gaze. These produce a transient {{XRInputSource}} with a {{targetRayMode}} set to {{transient-intent}}, treated as a [=primary action=]. +Another example would be intents from the operating system with input derived from sensitive information that cannot be exposed directly, such as interactions based on gaze. These produce a transient {{XRInputSource}} with a {{targetRayMode}} set to {{transient-pointer}}, treated as a [=primary action=]. [=Transient input sources=] are only present in the session's [=list of active XR input sources=] for the duration of the [=transient action=].