Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fundamental Accessibility: Modify conhost accessibility model #1352

Closed
3 tasks
cinnamon-msft opened this issue Jun 20, 2019 · 2 comments
Closed
3 tasks

Fundamental Accessibility: Modify conhost accessibility model #1352

cinnamon-msft opened this issue Jun 20, 2019 · 2 comments
Assignees
@carlos-zamora
Labels
Area-Accessibility Issues related to accessibility In-PR This issue has a related PR Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Needs-Tag-Fix Doesn't match tag requirements Product-Terminal The new Windows Terminal.
Milestone
Terminal v0.3

Comments

@cinnamon-msft
Copy link
Contributor

Summary of the new feature/enhancement

Modify the current conhost accessibility model for Windows Terminal usage and integration.

Proposed technical implementation details (optional)

  • Find where obsolete objects are used in accessibility model
  • Modify method signatures to take in what they specifically use, rather than entire obsolete objects
  • Hook up this UIA tree to top-level (Top-Level Accessibility: Set Up UIA Tree #1350)
@cinnamon-msft cinnamon-msft added Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Area-Accessibility Issues related to accessibility labels Jun 20, 2019
@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels Jun 20, 2019
@oising
Copy link
Collaborator

oising commented Jun 20, 2019

Is this an opportunity to improve #410 ? In particular the accessibility broadcasts that slow down all consoles when node is running.

@carlos-zamora carlos-zamora self-assigned this Jun 20, 2019
@carlos-zamora carlos-zamora added Product-Terminal The new Windows Terminal. and removed Needs-Tag-Fix Doesn't match tag requirements Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting labels Jun 20, 2019
@DHowett-MSFT
Copy link
Contributor

@oising probably not, sorry!

@carlos-zamora carlos-zamora mentioned this issue Jul 10, 2019
Merged
4 tasks
@carlos-zamora carlos-zamora added In-PR This issue has a related PR and removed In-PR This issue has a related PR labels Jul 12, 2019
@carlos-zamora carlos-zamora mentioned this issue Jul 17, 2019
Merged
9 tasks
@DHowett-MSFT DHowett-MSFT added the In-PR This issue has a related PR label Jul 25, 2019
carlos-zamora added a commit that referenced this issue Jul 29, 2019
@carlos-zamora
Accessibility: Set-up UIA Tree (#1691)
96496d8 
**The Basics of Accessibility**
- [What is a User Interaction Automation (UIA) Tree?](https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-tree-overview)
- Other projects (i.e.: Narrator) can take advantage of this UIA tree and are used to present information within it.
- Some things like XAML already have a UIA Tree. So some UIA tree navigation and features are already there. It's just a matter of getting them hooked up and looking right.

**Accessibility in our Project**
There's a few important classes...
regarding Accessibility...
- **WindowUiaProvider**: This sets up the UIA tree for a window. So this is the top-level for the UIA tree.
- **ScreenInfoUiaProvider**: This sets up the UIA tree for a terminal buffer.
- **UiaTextRange**: This is essential to interacting with the UIA tree for the terminal buffer. Actually gets portions of the buffer and presents them.

regarding the Windows Terminal window...
- **BaseWindow**: The foundation to a window. Deals with HWNDs and that kind of stuff.
- **IslandWindow**: This extends `BaseWindow` and is actually what holds our Windows Terminal
- **NonClientIslandWindow**: An extension of the `IslandWindow`

regarding ConHost...
- **IConsoleWindow**: This is an interface for the console window.
- **Window**: This is the actual window for ConHost. Extends `IConsoleWindow`

- `IConsoleWindow` changes:
  - move into `Microsoft::Console::Types` (a shared space)
  - Have `IslandWindow` extend it
- `WindowUiaProvider` changes:
  - move into `Microsoft::Console::Types` (a shared space)
- Hook up `WindowUiaProvider` to IslandWindow (yay! we now have a tree)

### Changes to the WindowUiaProvider
As mentioned earlier, the WindowUiaProvider is the top-level UIA provider for our projects. To reuse as much code as possible, I created `Microsoft::Console::Types::WindowUiaProviderBase`. Any existing functions that reference a `ScreenInfoUiaProvider` were virtual-ized.

In each project, a `WindowUiaProvider : WindowUiaProviderBase` was created to define those virtual functions. Note that that will be the main difference between ConHost and Windows Terminal moving forward: how many TextBuffers are on the screen.

So, ConHost should be the same as before, with only one `ScreenInfoUiaProvider`, whereas Windows Terminal needs to (1) update which one is on the screen and (2) may have multiple on the screen.

🚨 Windows Terminal doesn't have the `ScreenInfoUiaProvider` hooked up yet. We'll have all the XAML elements in the UIA tree. But, since `TermControl` is a custom XAML Control, I need to hook up the `ScreenInfoUiaProvider` to it. This work will be done in a new PR and resolve GitHub Issue #1352.


### Moved to `Microsoft::Console::Types`
These files got moved to a shared area so that they can be used by both ConHost and Windows Terminal.
This means that any references to the `ServiceLocator` had to be removed.

- `IConsoleWindow`
  - Windows Terminal: `IslandWindow : IConsoleWindow`
- `ScreenInfoUiaProvider`
  - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details.
- `UiaTextRange`
  - all references to `ServiceLocator` and `SCREEN_INFORMATION` were removed. `IRenderData` was used to accomplish this. Refer to next section for more details.
  - since most of the functions were `static`, that means that an `IRenderData` had to be added into most of them.


### Changes to IRenderData
Since `IRenderData` is now being used to abstract out `ServiceLocator` and `SCREEN_INFORMATION`, I had to add a few functions here:
- `bool IsAreaSelected()`
- `void ClearSelection()`
- `void SelectNewRegion(...)`
- `HRESULT SearchForText(...)`

`SearchForText()` is a problem here. The overall new design is great! But Windows Terminal doesn't have a way to search for text in the buffer yet, whereas ConHost does. So I'm punting on this issue for now. It looks nasty, but just look at all the other pretty things here. :)
@ghost ghost added the Needs-Tag-Fix Doesn't match tag requirements label Jul 31, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@carlos-zamora carlos-zamora

Labels
Area-Accessibility Issues related to accessibility In-PR This issue has a related PR Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Needs-Tag-Fix Doesn't match tag requirements Product-Terminal The new Windows Terminal.
Projects
None yet
Milestone
Terminal v0.3
Development

No branches or pull requests
4 participants
@oising @carlos-zamora @DHowett-MSFT @cinnamon-msft