Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow selection in the a11y tree and sync the selection to terminal #4742

Merged
merged 4 commits into from
Nov 3, 2023

Conversation

JasonXJ
Copy link

@JasonXJ JasonXJ commented Sep 1, 2023

Fixes #4656

@JasonXJ
Copy link
Author

JasonXJ commented Sep 1, 2023

Hi @Tyriar, can you take a look at this. A few notes:

  • I put a const debug in AccessibilityManager.ts so that I can easily use the mouse to interact with the tree during development. I think it is quite useful so I leave it in the code. I am also happy to remove it.
  • The code in AccessibilityManager._handleSelectionChange() to "translate" the Selection object to a row-column pair is quite complicated, so I do want to write some tests there. But there isn't existing tests for the manager, and setting everything up for testing is a lot of work. So I am thinking about extracting some of the code to static methods and do unit tests on the static methods instead. What do you think about this?

@JasonXJ
Copy link
Author

JasonXJ commented Sep 27, 2023

Hi @Tyriar, can you take a look at this? Or can you let me know how I can help to get this merge quicker?

@Tyriar Tyriar added this to the 5.4.0 milestone Oct 2, 2023
@Tyriar Tyriar self-assigned this Oct 2, 2023
Jason Lin added 2 commits October 11, 2023 16:23
Previously, for string 'ab好', outColumns=[0, 1, 2]. Now it becomes [0,
1, 2, 4]. This allows the user to know where does the last character
ends.
Copy link
Member

@Tyriar Tyriar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work 👍

For tests, yes it is a little tricky to get them set up for this as it's quite entangled with the main Terminal object (cyclic dependency). If you want to do a follow up for tests which would be greatly appreciated, you could just turn this into an exported function and test that, instead of trying to test the method on AccessibilityManager.

@Tyriar
Copy link
Member

Tyriar commented Nov 3, 2023

FYI @meganrogge, this should allow screen readers to select text in the underlying textarea. I haven't tested it much so we'll want to keep an eye out in Insiders.

@Tyriar Tyriar merged commit 58bb7fc into xtermjs:master Nov 3, 2023
19 checks passed
Comment on lines +539 to +543
if (outColumns) {
for (let i = 0; i < chars.length; ++i) {
outColumns.push(startCol);
}
}
Copy link
Member

@jerch jerch Nov 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just stumbled over this code change and blamed it back to here.

This looks like a serious perf smell to me - note that this function is hot-path for handlers grabbing line contents over and over.
Ideally hot-path functions are as branching/inner-loop free as possible, and never change the inner works/touched variables to not get de-optimized from JIT. This change adds all of that.

Note that I have not perf tested it yet, so its just a "looks-alike" concern from my side atm.

@Tyriar FYI

Copy link
Author

@JasonXJ JasonXJ Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I played with it a bit and this did not seem to be called much during normal rendering. It is called quite a lot by the SearchAddon (when you are using search) though, but it is unclear to me whether it actually affect performance.

Copy link
Member

@jerch jerch Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that method is not called during normal data ingestion & rendering (basic operations). But, as you already found, it gets used by several 2nd-line operations, which do buffer scraping alot, like search, serialize, in general things operating on the buffer API to grab the text repr. While those actions are rarely triggered, they most likely run over bigger bufferlines batches creating some runtime pressure for a short amount of time (potentially leading to hiccups).

Will see if I find the time to perf it - tricky part here prolly is, that the de-opt might only show up, if calls alternate between outColumns being set and unset. Not sure yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to select text with screen reader
3 participants