Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Shifted symbols aren't announced in input help #6621

Open
Qchristensen opened this issue Dec 6, 2016 · 31 comments
Open

Shifted symbols aren't announced in input help #6621

Qchristensen opened this issue Dec 6, 2016 · 31 comments
Labels
enhancement p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority

Comments

@Qchristensen
Copy link
Member

When using input help mode, NVDA+1, NVDA does not read the character generated by pressing shift+key. For instance, on an English-US keyboard, the dollar sign is shift+4. Press this using input help and NVDA only reads "shift+4". It would be useful to have it read the symbol as well as the keys.
@gregjozk
Copy link
Contributor

gregjozk commented Dec 8, 2016

same behaveour is expected when ctrl + alt + symbol (number, letter or other char) on the keyboard is pressed.

@feerrenrut
Copy link
Contributor

@Qchristensen can you clarify the expected output for shift+4. I am interpreting this, as outputting something like "shift+4, $". Which may announce as "shift plus 4, dollar" (depending on your settings). Is this what you were thinking?

Maybe this is an edge case we should just ignore, but I think this could take us down a tricky path. We are announcing the input keys, and then the "effect" that the particular combination of keys results in. This is simple for shift + symbolic keys resulting in the symbol, or shift + letters resulting in the capital letter. However when you include ctrl and alt, this can start to get quite specific. For instance, in the case of ctrl + c do we then report "copy", this may not apply in some contexts.

@Qchristensen
Copy link
Member Author

Qchristensen commented Dec 8, 2016 via email

@gregjozk
Copy link
Contributor

gregjozk commented Dec 8, 2016 via email

@Qchristensen
Copy link
Member Author

Ah ok, well I wonder @feerrenrut can we find out whether a key combination produces a symbol and announce it (as I was thinking for things like shift+4 $ but clearly can use other keys in different language layouts)?

@jcsteh
Copy link
Contributor

jcsteh commented Dec 8, 2016

there are a few reasons I'm not sure we can/should do this:

  1. In some languages, it's not possible to determine a character from one keystroke. There are "dead keys" where the first key does nothing and the second produces a character different to that which would be produced if the first weren't pressed. I'm not sure this will work unless we allow the first character to be passed to the system, but we obviously want to intercept and block it in input help.
  2. The character might be different depending on the focused application, since (at least in some versions of Windows) the keyboard layout can be different across apps.3. The actual character is often not how we want to explain the command to users. For example, in browse mode, shifted letters quick navigate backwards, but we think of them as, for example, shift+h, not capital H.

@feerrenrut
Copy link
Contributor

It might be worth exploring the use case for this feature a little more. I interpreted the use case to be something like "As a computer user who is unfamiliar with the keyboard layout I would like to be able to explore the keys on the keyboard so that I can discover what various key combinations output". Sighted users can see the markings on the keyboard to work out what certain keys do, however we dont have a solution for this when it comes to the symbols.

@Qchristensen
Copy link
Member Author

Reef, that was essentially the use case I had in mind. I must admit I hadn't thought of the various ways different language keyboards produce symbols aside from shift+key.

I wonder then, following on from that, on the Slovenian keyboard, is there any visual representation that control+alt+v inputs a @ symbol? Now that I think about it, I do recall seeing at least one non-English keyboard which had the letter at the bottom of the key, a symbol top left and another symbol top right which presumably was accessed via some other keystroke... Ok, here's a page talking about (and an image of) the Slovenian keyboard layout: http://smotko.si/coding-and-keyboard-layouts/

And here's another showing a Lenovo Thinkpad Slovenian Croatian keyboard:
http://www.keysourcechina.com/Lenovo-ThinkPad-T400S-T410-X220-Croatian-Keyboard.html

So, clearly it's more complex than I'd originally anticipated. Jamie's roadblocks sound fairly solid so I think it will be tricky to implement without a way around those, unless it was only done either for certain keys (eg shift+key) or for certain languages. I think if we did shift+key, while that would suit the majority of simple English keyboards, it would then be confusing for other language keyboards where some keystrokes suddenly get announced but not others.

@feerrenrut
Copy link
Contributor

Well in that case I think we understand what we want. It does seem like there are a few things to watch out for, but if we keep the use case in mind then application or OS specific functionality can be ignored. We are only trying to allow a user to explore what keys their keyboard can type. Handling multiple keystrokes will be harder, but perhaps we can defer support for that.

I'll set this as a priority 3, when we get to it we can do some further investigation into how hard it will be to implement.

@feerrenrut feerrenrut added the p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority label Dec 8, 2016
@Qchristensen
Copy link
Member Author

One use case for this I just encountered was a user who wrote to ask where the percent key was. (It's shift+5 for anyone wondering). While that fits the above case, I was thinking - on most keyboards, those "common" symbols that you can access by pressing a key or by pressing SHIFT+key are printed so a sighted user could look at the keyboard to find out which key to press. Input help is essentially NVDA's replacement of "a sighted user looking at the keyboard".

My Microsoft keyboard has as many extra keys as proper keys and it is interesting to note that they are reported in Input help ("Email", "Browser", "Skype" etc). That probably muddies the "key vs function" argument, but it does pretty much work (there are some which don't or which execute a function, even in input help mode) so I'll leave it alone :)

@Brian1Gaff
Copy link

Brian1Gaff commented Jun 16, 2020 via email •
edited by feerrenrut
Loading

@CyrilleB79 CyrilleB79 mentioned this issue Mar 28, 2022
Closed
@CyrilleB79 CyrilleB79 mentioned this issue Sep 20, 2023
Closed
@britechguy
Copy link

I'm the one who opened the now closed #15480 and in the topic on the NVDA Group I'd made reference to, NVDA Input Help and characters produced using SHIFT Quentin had commented that I should open an issue. He had clearly forgotten that he opened this one all the way back in 2016.

This should have been addressed long ago. Hearing "Shift plus whatever the non-shift key" happens to be for any key is not at all useful. When SHIFT plus is used to produce a separate character, that's what should be announced.

@CyrilleB79
Copy link
Collaborator

In this issue, we are actually discussing two features:

  • Input help: This allow to know if there is an NVDA command associated to a key combination.
  • Keyboard learning: This allows to know which character is written when typing a key combination

Input help is implemented in NVDA and keyboard learning is not. Thus some people use input help as a way to learn the keyboard but it's far from perfect.

It's not easy to merge the two features in NVDA's input help. Indeed, there are some combinations where there may be both an associated NVDA command and a typed character. E.g. in a browser, if I type shift+H, input help should report move to previous heading, but keyboard learning should report "Capital H".

Thus I would recommend rather to implement a separate keyboard learning feature and not to try to merge it with input help. This would help clarify the difference between an NVDA command and a typed character.

@britechguy
Copy link

Cyrille,

The problem being that if you're in a browser, with input help on, and hit H or SHIFT + H you hear just that. You do not hear jump to next heading or jump to previous heading.

I've never known the single key quick navigation commands to be described by Input Help. But if you're going to say "H" and "SHIFT plus H" that's not particularly useful, either. I'd rather have "H" and "Capital H."

Input Help is, in my opinion, a combination of keyboard learning and NVDA command learning, and only for NVDA commands that are multi-keystroke with the exception of the number-pad based single character commands. The Number Pad is a sort of "world of its own" that is treated as separate from the main keyboard.

@ruifontes
Copy link
Contributor

ruifontes commented Sep 20, 2023 via email

@seanbudd
Copy link
Member

Doesn't "Speak typed characters" handle the "keyboard learning" feature and this issue?

@ruifontes
Copy link
Contributor

ruifontes commented Sep 20, 2023 via email

@britechguy
Copy link

britechguy commented Sep 20, 2023 •
edited
Loading

Are you sure?

Clearly not. But I've also changed versions of NVDA since my prior test today. I was using NVDA 2023.2, and am now using NVDA 2023.3beta2. I was using the main page of the New York Times as my test location.

I am positive that earlier I was only getting H and SHIFT + H announced. Now I'm getting the letter and function, and it's telling me Left Shift not just shift.

I also am absolutely positive that yesterday, when working with a client, I tried hitting the numbers 1 through 3 or 4 on the number row with Input Help active and only got the numbers, nothing about moving to heading level 1 (2, 3, 4). That's what actually triggered me to ask about this on the NVDA group and to create the now closed as duplicate feature request.

@britechguy
Copy link

Nop, Unfortunately this feature do not work as advertised! It should announce printable characters, but, by instance, do not announce "´ accut accent", "` grav accent", "~ tilde" and "^ caret"...

Interesting, because when I punted to simply typing characters in Notepad yesterday, I know that both tilde and grave (oddly pronounced in English as though it were the start of the word gravel, which is still better than announcing it as though it were a burial spot).

I just turned on speak typed characters now, using NVDA 2023.3beta2 and it's announcing every shifted character on the number row when I type same in Notepad. Although why any synth (and I know it's the synth, not NVDA) calls the underscore character "line" eludes me.

@XLTechie
Copy link
Collaborator

XLTechie commented Sep 21, 2023 via email

@britechguy
Copy link

Well, if you have, say Notepad, open and focus where you can edit, for the keys you get key name, shift plus keyname, with Input Help active, which is what I have to believe I was doing yesterday.

I still think that shift plus 1, in that context, should say "bang." Etc., etc., etc.

@XLTechie
Copy link
Collaborator

XLTechie commented Sep 21, 2023 via email

@CyrilleB79
Copy link
Collaborator

But @CyrilleB79, when there is no NVDA command associated, what is the value of indicating "Shift+1" instead of (in English) "Exclamation point"? I get what you're saying about browse mode changes, but when there is no conflict, what is the harm?

There is no harm when there is no conflict.
The issue is just that if you are in input help mode,, you may hear "Jump to end of container" when looking for comma on the keyboard. That's why I am advocating for a separate feature.

@britechguy
Copy link

I agree with @CyrilleB79 that a completely separate "Keyboard Learning" mode is a great idea. Wanting to know "what's where" on the keyboard as a whole is a separate issue from wanting to know what key or key combinations make NVDA do something.

That being said, for those SHIFT plus key combinations that are not NVDA commands, when Input Help is on they should be announced as the character that SHIFT plus key would produce, not SHIFT plus the lowercase option on that key.

@CyrilleB79
Copy link
Collaborator

That being said, for those SHIFT plus key combinations that are not NVDA commands, when Input Help is on they should be announced as the character that SHIFT plus key would produce, not SHIFT plus the lowercase option on that key.

I disagree. In NVDA command learning mode, it would be clearer to indicate "No an NVDA command" when there is no associated command to be extra-clear. This way, users would know that NVDA+F12 is a command executed by NVDA, but that printing "$" is not done by NVDA but by the application.

Also, some specificities to keep in mind:

  • what is reported in input help mode when pressing a key is its name, i.e. the most significant character on the key. Usually, it's the unshifted character but not always. A counter-example are number keys on French keyboard. The key named "1" writes "&" when shift is not pressed and "1" when shift is pressed; said otherwise shift is required to write digits on the alpha-numeric keyboard. Though, in input help mode, NVDA reports "1" when this key is pressed without shift and "majuscule+1" (that is "shift+1") when shift is pressed.
  • Also some keys do not produce any character when combined with shift. E.g. on the French keyboard, the key "²" is just above the Tab key; but shift+² does not produce anything (and nothing else is written on the key).
  • On many non-English keyboards, the right alt key (on the right of space bar) is replaced by another modifier: AltGr in French. It allows to write some more characters, but as for the "²" key, there are many keys with no combination including AltGr.

All these specificities let me think that it would be quite hard to develop a bug-free keyboard learning feature. In NVDA command learning feature, if we use the keyboard learning feature output for combination that do not correspond to an NVDA commands, I feel it would just add confusion.

For reference, in input help mode, Narrator reports the character when there is no associated script as you request. And it behaves very poorly with the specific cases that I have presented:

  • shift+² indicates "Unknown key"
  • AltGr+1 reports "&" as if shift were not pressed.

@Adriani90
Copy link
Collaborator

In NVDA command learning feature, if we use the keyboard learning feature output for combination that do not correspond to an NVDA commands, I feel it would just add confusion.

I agree.

Regarding keyboard learning feature, many instances of keys are already announced when enabling "speak typed characters" and "speak modifier keys". I guess this is already a good basis on which the two settings could be extended to report more keys which are not covered yet. I think it should not be hard to bundle them and a keyboard command could activate both settings at once while NVDA reports "keyboard learning enabled" or "disabled" on the second press.

@britechguy
Copy link

In NVDA command learning mode, it would be clearer to indicate "No an NVDA command" when there is no associated command to be extra-clear.

Which would be just fine, but you are never going to get me to agree that saying, "Shift + lowercase character on key" is more appropriate than saying the character produced by that shifted keypress. I also am not sold on adding that explicit announcement about something not being a command.

Right now it is assumed that if something does not have an NVDA command description when pressed, then it's not an NVDA command. I don't think that having everything that's not an NVDA command announcing that fact necessarily is "value added," and it strikes me as excess verbiage.

If users have been expected to know, up to this point, that something's not an NVDA command unless Input Help announces that it is, that's a reasonable expectation going forward.

@CyrilleB79
Copy link
Collaborator

Which would be just fine, but you are never going to get me to agree that saying, "Shift + lowercase character on key" is more appropriate than saying the character produced by that shifted keypress. I also am not sold on adding that explicit announcement about something not being a command.

OK, it's your opinion; and mine is different.

And for completeness, what do you suggest to be reported when the key combination produces no character? E.g. "shift+²" as described in #6621 (comment). Note the answer to this question may be especially useful for keyboard learning feature.

Right now it is assumed that if something does not have an NVDA command description when pressed, then it's not an NVDA command. I don't think that having everything that's not an NVDA command announcing that fact necessarily is "value added," and it strikes me as excess verbiage.

If users have been expected to know, up to this point, that something's not an NVDA command unless Input Help announces that it is, that's a reasonable expectation going forward.

That's right, I agree with your conclusion on this point. Users have never complained of any ambiguity when no command is reported.

@britechguy
Copy link

And for completeness, what do you suggest to be reported when the key combination produces no character? E.g. "shift+²" as described in #6621 (comment). Note the answer to this question may be especially useful for keyboard learning feature.

I have absolutely no idea, as neither of those combinations exist on a standard U.S. English keyboard. I am aware of ALT Gr, but those who actually use keyboards with the keys (and combinations) mentioned should know what the expected output would be.

And if that would be "no output" when you strike those combinations, then something like "No typed output," should be announced.

Again, this comes back to specific language communities knowing what character is produced when a given key combination is struck on the keyboards common in those areas. I can't speak to what NVDA should say for languages I literally don't speak and for key combinations that do not exist with the keyboard I am using.

@XLTechie
Copy link
Collaborator

XLTechie commented Sep 23, 2023 via email

@britechguy
Copy link

I think it should do that, although Cyrille raises some concerning points about non-English keyboards.

And I think I addressed the major concern directly, and, in fact, so have you. There is no "universally correct" thing to say, as keyboards differ, but there is a virtually universally correct thing to do, and what that is has been expressed repeatedly, and concisely.

If hitting Modifier + Key on some keyboard produced a character called goorgleformps, then what the screen reader should announce in Input Help Mode would depend on whether that key combination was a command for the screen reader or not. If it is, then describe it as "Modifier plus key, screen reader function," if it's not, describe it as "Goorgleformps." This is a general principle not tied to any keyboard or language in particular.

I really don't find it helpful to have constant announcement of "Not a screen reader command." If something is, then that's noted and announced. If it's not, then the key combination is announced.

This really is not rocket science, and I can't believe how many circles this discussion has gone in. It's been chasing its own tail for quite a few comments now. There are not myriad options. Either change from what happens now, or don't. But the "how" of the change has been articulated many times.

@CyrilleB79 CyrilleB79 mentioned this issue Dec 8, 2023
Open
@Emil-18 Emil-18 mentioned this issue Dec 12, 2023
Closed
5 tasks
@cary-rowen cary-rowen mentioned this issue Jan 16, 2025
Draft
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests