Feature request: Disabling bidi support

[hseg]

TL;DR: Some applications (e.g. vim) have their own bidi support, which conflicts
with kitty's support. Please offer a way of turning off kitty's bidi support.

Kitty's bidi seems to cut the line into words, shape each word, then place them
in one line left-to-right, whereas vim's bidi just places letters one after the
other in the specified direction.
(Side note: I can't find kitty's behaviour here documented anywhere)

Hence, kitty's bidi works reasonably well for interactive shell use and the
like, where there is a prevailing directionality and the reversed text is
relatively rare. Conversely, vim's bidi works well in longform text where
changes in direction occur in multi-word streaks.

(Hypothetically, if a good bidi algorithm were implemented (e.g.
UAX9), it would deprecate vim's bidi in
some scenarios, but not all – in cases where extra markup exists, such
algorithms would probably get confused by the markup and give wrong results)

However, kitty+vim is broken in this regard – kitty lies to vim about what's
under the cursor, editing longform text written in this zigzag layout is tiring,
and I don't even have the usual vim workaround of opening the file and its
reflection, since then the words are also reflected.

[kovidgoyal]

dup of #2109

[hseg]

Not a dup - am asking for doc + config for a feature already existing in kitty today.

…

On 5 January 2020 5:28:43 GMT+02:00, Kovid Goyal ***@***.***> wrote: Closed #2252. -- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #2252 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[kovidgoyal]

There is no bidi supportin kitty at all. What you see if just harfbuzz
reversing rtl text based on unicode code points.

[hseg]

Ah. So implementing this would be a matter of toggling between HarfBuzz and a naive shaper, the development effort of which would be better served improving the bidi support. I will submit, though, that HarfBuzz-style shaping is inappropriate when either the underlying application does its own reordering, or when one wants to see RR he text in logical order (eg when debugging compilation errors). Only thing that's unclear is why bidi text is displayed in reverse word order, ie the equivalent of WORLD HELLO. Is kitty splitting words before passing them?

…

On 5 January 2020 12:42:49 GMT+02:00, Kovid Goyal ***@***.***> wrote: There is no bidi supportin kitty at all. What you see if just harfbuzz reversing rtl text based on unicode code points. -- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #2252 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[kovidgoyal]

On Sun, Jan 05, 2020 at 04:57:08AM -0800, hseg wrote: Ah. So implementing this would be a matter of toggling between HarfBuzz and a naive shaper, the development effort of which would be better served improving the bidi support. I will submit, though, that HarfBuzz-style shaping is inappropriate when either the underlying application does its own reordering, or when one wants to see RR he text in logical order (eg when debugging compilation errors).

Yeah and proper bidi support means having some way for the application to inform the terminal what level of bidi support it needs/wants. Without that its just whack-a-mole.

Only thing that's unclear is why bidi text is displayed in reverse word order, ie the equivalent of WORLD HELLO. Is kitty splitting words before passing them?

Yup, kitty rendering fundamentally works at a character cell level, not at the level of words/lines/whatever other logical unit you can name. And character cell rendering works on runs, runs that are broken up at spaces, font changes, various other things. This allows for many optimisations that give it its stellar performance, but it means support for complex scripts/bidi/etc. is never going to be that good.

[hseg]

On 5 January 2020 15:04:31 GMT+02:00, Kovid Goyal ***@***.***> wrote: On Sun, Jan 05, 2020 at 04:57:08AM -0800, hseg wrote: > Ah. So implementing this would be a matter of toggling between HarfBuzz and a naive shaper, the development effort of which would be better served improving the bidi support. I will submit, though, that HarfBuzz-style shaping is inappropriate when either the underlying application does its own reordering, or when one wants to see RR he text in logical order (eg when debugging compilation errors). Yeah and proper bidi support means having some way for the application to inform the terminal what level of bidi support it needs/wants. Without that its just whack-a-mole.

Which is why I suggested toggling shaping - that way I can at least wrap the relevant programs.

> Only thing that's unclear is why bidi text is displayed in reverse word order, ie the equivalent of WORLD HELLO. Is kitty splitting words before passing them? Yup, kitty rendering fundamentally works at a character cell level, not at the level of words/lines/whatever other logical unit you can name. And character cell rendering works on runs, runs that are broken up at spaces, font changes, various other things. This allows for many optimisations that give it its stellar performance, but it means support for complex scripts/bidi/etc. is never going to be that good.

Oh... and I suppose joining runs separated by weak characters would be problematic for that? Just that would already be nice (with the caveat that lines should be processed LTR by default due to mostly-English command line) -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[kovidgoyal]

On Sun, Jan 05, 2020 at 05:46:24AM -0800, hseg wrote: On 5 January 2020 15:04:31 GMT+02:00, Kovid Goyal ***@***.***> wrote: >On Sun, Jan 05, 2020 at 04:57:08AM -0800, hseg wrote: >> Ah. So implementing this would be a matter of toggling between >HarfBuzz and a naive shaper, the development effort of which would be >better served improving the bidi support. I will submit, though, that >HarfBuzz-style shaping is inappropriate when either the underlying >application does its own reordering, or when one wants to see RR he >text in logical order (eg when debugging compilation errors). > >Yeah and proper bidi support means having some way for the application >to inform the terminal what level of bidi support it needs/wants. >Without that its just whack-a-mole. Which is why I suggested toggling shaping - that way I can at least wrap the relevant programs.

Yes, but that requires a fair amount of work and is just a bandaid. If I am going to put in the work, I am not going to do it for a bandaid. I encourage you to contribute code to kitty for bidi support i it is important to you, personally, its not a high priority.

>> Only thing that's unclear is why bidi text is displayed in reverse >word order, ie the equivalent of WORLD HELLO. Is kitty splitting words >before passing them? > >Yup, kitty rendering fundamentally works at a character cell level, not >at the level of words/lines/whatever other logical unit you can name. >And character cell rendering works on runs, runs that are broken up at >spaces, font changes, various other things. > >This allows for many optimisations that give it its stellar >performance, >but it means support for complex scripts/bidi/etc. is never going to be > >that good. Oh... and I suppose joining runs separated by weak characters would be problematic for that? Just that would already be nice (with the caveat that lines should be processed LTR by default due to mostly-English command line)

It might not be, depends on the details, and I dont understand bidi well enough to comment authoritatively.

[hseg]

OK, thanks. Have a workaround ATM (use gvim), but will put this on the back burner for if this gets annoying enough before you fix bidi.

…

On 5 January 2020 16:32:28 GMT+02:00, Kovid Goyal ***@***.***> wrote: On Sun, Jan 05, 2020 at 05:46:24AM -0800, hseg wrote: > > > On 5 January 2020 15:04:31 GMT+02:00, Kovid Goyal ***@***.***> wrote: > >On Sun, Jan 05, 2020 at 04:57:08AM -0800, hseg wrote: > >> Ah. So implementing this would be a matter of toggling between > >HarfBuzz and a naive shaper, the development effort of which would be > >better served improving the bidi support. I will submit, though, that > >HarfBuzz-style shaping is inappropriate when either the underlying > >application does its own reordering, or when one wants to see RR he > >text in logical order (eg when debugging compilation errors). > > > >Yeah and proper bidi support means having some way for the application > >to inform the terminal what level of bidi support it needs/wants. > >Without that its just whack-a-mole. > > Which is why I suggested toggling shaping - that way I can at least wrap the relevant programs. Yes, but that requires a fair amount of work and is just a bandaid. If I am going to put in the work, I am not going to do it for a bandaid. I encourage you to contribute code to kitty for bidi support i it is important to you, personally, its not a high priority. > > >> Only thing that's unclear is why bidi text is displayed in reverse > >word order, ie the equivalent of WORLD HELLO. Is kitty splitting words > >before passing them? > > > >Yup, kitty rendering fundamentally works at a character cell level, not > >at the level of words/lines/whatever other logical unit you can name. > >And character cell rendering works on runs, runs that are broken up at > >spaces, font changes, various other things. > > > >This allows for many optimisations that give it its stellar > >performance, > >but it means support for complex scripts/bidi/etc. is never going to be > > > >that good. > > Oh... and I suppose joining runs separated by weak characters would be problematic for that? Just that would already be nice (with the caveat that lines should be processed LTR by default due to mostly-English command line) It might not be, depends on the details, and I dont understand bidi well enough to comment authoritatively. -- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #2252 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[hseg]

Thanks!

…

On 5 January 2020 16:32:28 GMT+02:00, Kovid Goyal ***@***.***> wrote: On Sun, Jan 05, 2020 at 05:46:24AM -0800, hseg wrote: > > > On 5 January 2020 15:04:31 GMT+02:00, Kovid Goyal ***@***.***> wrote: > >On Sun, Jan 05, 2020 at 04:57:08AM -0800, hseg wrote: > >> Ah. So implementing this would be a matter of toggling between > >HarfBuzz and a naive shaper, the development effort of which would be > >better served improving the bidi support. I will submit, though, that > >HarfBuzz-style shaping is inappropriate when either the underlying > >application does its own reordering, or when one wants to see RR he > >text in logical order (eg when debugging compilation errors). > > > >Yeah and proper bidi support means having some way for the application > >to inform the terminal what level of bidi support it needs/wants. > >Without that its just whack-a-mole. > > Which is why I suggested toggling shaping - that way I can at least wrap the relevant programs. Yes, but that requires a fair amount of work and is just a bandaid. If I am going to put in the work, I am not going to do it for a bandaid. I encourage you to contribute code to kitty for bidi support i it is important to you, personally, its not a high priority. > > >> Only thing that's unclear is why bidi text is displayed in reverse > >word order, ie the equivalent of WORLD HELLO. Is kitty splitting words > >before passing them? > > > >Yup, kitty rendering fundamentally works at a character cell level, not > >at the level of words/lines/whatever other logical unit you can name. > >And character cell rendering works on runs, runs that are broken up at > >spaces, font changes, various other things. > > > >This allows for many optimisations that give it its stellar > >performance, > >but it means support for complex scripts/bidi/etc. is never going to be > > > >that good. > > Oh... and I suppose joining runs separated by weak characters would be problematic for that? Just that would already be nice (with the caveat that lines should be processed LTR by default due to mostly-English command line) It might not be, depends on the details, and I dont understand bidi well enough to comment authoritatively. -- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #2252 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.