[Feature Request] - Implement kittys terminal graphics protocol

[misaelaguayo]

It would be neat if zellij could use kitty terminal graphics to display images or other files. Tmux had a similar issue closed, it seems like they will not be implementing it. Zellij does provide support for sixels but this is a very old protocol with not much support from other applications.

[tcoopman]

FIY: On the yazi wiki, there's a fix to use it with tmux: https://github.com/sxyazi/yazi/wiki/Image-preview-within-tmux
This looks like a workaround, not sure how it works exactly but maybe zellij can do something similar?

[daveman1010221]

Is there a sense of the work required to implement this?

[Dich0tomy]

FIY: On the yazi wiki, there's a fix to use it with tmux: https://github.com/sxyazi/yazi/wiki/Image-preview-within-tmux This looks like a workaround, not sure how it works exactly but maybe zellij can do something similar?

Yes, but it's extremely slow on tmux for some reason.

[Dich0tomy]

Is there a sense of the work required to implement this?

There's a plethora of users that want to enjoy images in their terminal and some terminal tools implement image support (image.nvim plugin for neovim, neorg plugin for neovim, a lot of file browsers, etc.). I almost never leave terminal and currently use tmux, image implementation in zellij would be a dealbreaker for me.

[sxyazi]

I really want to see Zellij support kitty graphics protocol, which is currently the most complete and advanced.

It supports Unicode placeholders, I think this could solve the stuttering issue in Zellij's Sixel implementation. Kitty protocol simply prints some escape sequence to the screen, and terminals that support that protocol display it as an image and accurately render the image position when scrolling.

This process is transparent to Zellij, and it doesn't need to parse and understand the meaning of the protocol itself, making it efficient.

I've just implemented Unicode placeholders for Yazi and tested it in kitty+tmux, it works well:

kitty+tmux.mp4

[musjj]

@sxyazi
Is there something that needs to be done on zellij's side for this to work? I'm testing yazi (with the latest commit) from inside zellij, but image previews still doesn't work.

I've also tested it under tmux and it kinda works, but everything is really buggy (overlapping images, preview not updating, etc.) for some reason.

[sxyazi]

Is there something that needs to be done on zellij's side for this to work? I'm testing yazi (with the latest commit) from inside zellij, but image previews still doesn't work.

Yes, Zellij does not support it; this requires Zellij to provide support.

I've also tested it under tmux and it kinda works, but everything is really buggy (overlapping images, preview not updating, etc.) for some reason.

What are your Yazi version and tmux version? What terminal are you using? Please also do an echo $TERM, $TERM_PROGRAM, $XDG_SESSION_TYPE and paste the output here.

[AnonymouX47]

Just came across this and thought I should chime in.

I believe what's required on zellij's end is a Passthrough sequence (I don't know if such exists yet but I suspect not) i.e some something like this. No special handling of graphics control sequences should be required.

This should be all that's required to get Kitty's virtual placements working, assuming Zellij already supports indexed- (and optionally, direct-) -color SGR sequences and passes them through as recieved.

It'll also allow the use of other image/graphics protocols to a certain extent, particularly in full-screen (non-scrolling) applications. For full/proper support, specialized implementations/handling will be required but personally, I'd discourage that for a terminal multiplexer.

[sweetbbak]

This is also how this issue was solved with junegunn/fzf junegunn/fzf kitty image protocol passthrough fix

He uses a regex to detect the terminal sequence for images then assigns a no-op function for the preview that allows the un-altered terminal sequence to passthrough to the terminal so that the image is rendered like normal.

You can test this with:

fd . ~/Pictures -e png | fzf --preview='kitty icat --clear --transfer-mode=memory --stdin=no --place=${FZF_PREVIEW_COLUMNS}x${FZF_PREVIEW_LINES}@0x0 {}'

[hfytr]

is this still being worked on?

[misaelaguayo]

I don’t think anyone has started to implement this. Just been talking about ways in which it could be achieved

[misaelaguayo]

I've been digging into this a bit more since I would like to see it implemented. It seems that zellij uses alacritty's vte parser to interpret ANSI/VT codes in terminal_pane.rs. Perhaps in the handle_pty_bytes function we can do a similar approach to what was done for fzf and do a no op and not pass in those bytes to the vte_parser

[quadespresso]

Fwiw I've tried the following:

• Kitty + yazi ✔️
• Kitty + tmux + yazi ✔️
• Kitty + Zellij + yazi ❌
• Wezterm + yazi ✔️
• Wezterm + tmux + yazi ✔️
• Wezterm + Zellij + yazi ✔️

❯ neofetch
██████████████████  ████████   jimc@kapua 
██████████████████  ████████   ---------- 
██████████████████  ████████   OS: Manjaro Linux x86_64 
██████████████████  ████████   Host: NH5x_NH7xHP 
████████            ████████   Kernel: 6.1.99-1-MANJARO 
████████  ████████  ████████   Uptime: 10 days, 15 hours, 22 mins 
████████  ████████  ████████   Packages: 3373 (pacman), 75 (nix-user), 48 (nix-default) 
████████  ████████  ████████   Shell: zsh 5.9 
████████  ████████  ████████   Resolution: 1920x1080 
████████  ████████  ████████   DE: GNOME 46.3.1 
████████  ████████  ████████   WM: Mutter 
████████  ████████  ████████   WM Theme: Custom-Accent-Colors 
████████  ████████  ████████   Theme: Adw-dark [GTK2/3] 
████████  ████████  ████████   Icons: Papirus-Dark-Maia [GTK2/3] 
                               Terminal: kitty 
                               CPU: 11th Gen Intel i7-11800H (16) @ 4.600GHz 
                               GPU: Intel TigerLake-H GT1 [UHD Graphics] 
                               GPU: NVIDIA GeForce RTX 3060 Mobile / Max-Q 
                               Memory: 41611MiB / 64087MiB 

[merikan]

@quadespresso That's great news, I didn't know it worked in Wezterm. Thank you for looking into this 👍

I've been using Alacritty for years and really missed the support for displaying images in the terminal and also ligatures. A few weeks ago decided to switch to Kitty and was somewhat surprised that images were not rendered in Zellij. After reading your comment I switched to Wezterm and everything works perfectly. 🥰

A funny thing was that when I, in Wezterm, attached to a running session started with Kitty, the images were not displayed. I had to close the session and restart it.

$ fastfetch

OS: macOS Sonoma 14.5 arm64                                              
Host: MacBook Pro (16-inch, 2023)                                        
Kernel: Darwin 23.5.0                                                    
Uptime: 22 hours, 26 mins                                                
Packages: 42 (nix-default), 412 (brew), 133 (brew-cask)                  
Shell: zsh 5.9                                                           
Display (DELL U3419W): 3440x1440 @ 60 Hz in 34″ [External] *             
Display (Color LCD): 3456x2234 @ 120 Hz (as 1728x1117) in 16″ [Built-in] 
DE: Aqua                                                                 
WM: Quartz Compositor                                                    
WM Theme: Multicolor (Dark)                                              
Font: .AppleSystemUIFont [System], Helvetica [User]                      
Cursor: Fill - Black, Outline - White (32px)                             
Terminal: zellij 0.40.1                                                  
CPU: Apple M2 Max (12) @ 3,50 GHz                                        
GPU: Apple M2 Max (30) @ 1,40 GHz [Integrated]                           
Memory: 33,45 GiB / 64,00 GiB (52%)                                      
Swap: Disabled                                                           
Disk (/): 649,98 GiB / 926,35 GiB (70%) - apfs [Read-only]                                                   
Battery: 100% [AC connected]                                             
Power Adapter: 90W                                                       
Locale: en_US.UTF-8                                                      

[quadespresso]

@merikan interesting test scenario.

Based on that, I tried the opposite: I ran wezterm -> zellij -> yazi then navigated to an image to view.

Then I detached from the zellij session, reconnecting from a kitty terminal. It was a bit of a mess, an entire screenful of characters. Nothing vaguely resembling yazi was to be seen.

[merikan]

@quadespresso
That was interesting. I had to try it too and can confirm that it just gets a bunch of characters all over the screen. A little surprising that it is not only the image on the right side that gets distorted, but also the directory structures etc.

I couldn't resist, but also had Alacritty connected to the same session and there everything looked ok, except that the image is not displayed as we already know.

Kind of fun to look at multiple clients connected to the same session when jumping around in Yazi 😁

[quadespresso]

@merikan it's always interesting to cross-check with alternatives. I ended up trying the same combo with Alacritty and saw the same, ie, yazi doesn't display the image, but is otherwise intact.

Had a quick go with a few other terminal emulators (Terminator, Terminology, GNOME Terminal, Tilix), but only got as far as doing a quick check with yazi (minus zellij) to verify it couldn't display images before proceeding beyond that. I couldn't recall whether any of those supported inline images, and figured this would be quicker than pulling up docs for each.

[hongquan]

@quadespresso Which version of "Wezterm + Zellij + yazi" that can display image?
My version doesn't work:

• WezTerm 20240203-110809-5046fc22
• Zellij 0.40.1
• Yazi 0.3.1

[sweetbbak]

None. Zellij would have to implement some form of a passthrough ansi sequence that wraps image protocol sequences that indicate that the sequence should be let through the multiplexer. tmux/wiki/FAQ#what-is-the-passthrough-escape-sequence-and-how-do-i-use-it Im pretty sure zellij just strips the escape code which leaves a mess of base64 image data

currently icat for the Image kitty protocol automatically detects if its inside of tmux and uses unicode placeholder data and the special tmux ansi escape to wrap its encoded image data.

[quadespresso]

@hongquan here we go:

• WezTerm 20240203.110809.5046fc22-2
• Zellij 0.40.1-1
• Yazi 0.2.5-2

What's odd is that as I tested this today, I got nothing when I navigated yazi to the image. Blank in the area where I expected to see the image,

I then quit yazi, quit zellij, started yazi, verified that I could see the image, quit yazi, started zellij and then yazi, and then I could see the image without issue.

[merikan]

@quadespresso
I have the same inconsistent behavior. I have two long running zellij sessions and when I open a new pane and run yazi, only one of them render the image. 🤔. If I start a new session everything works ok.
I am trying to understand what is different between the two sessions but they are both started at about the same time and have version 0.40.1.
Killing and restarting the "non-working" session, in the same wezterm window, did not help. If I kill and restart the "working" session, in the same wezterm window, it still works.
If I kill and restart the the "non-working" session in a new wezterm window it starts rendering images. 🥳

zellij 0.40.1
wezterm 20240203-110809-5046fc22
Yazi 0.3.2 (Homebrew 2024-08-28)

[sxyazi]

I found this in the Zellij Discord server might be helpful (Quote from @imsnif in https://discord.com/channels/771367133715628073/771367133715628076/1272810902458204191):

Hey - Zellij supports Sixel for displaying images in the terminal (assuming your terminal emulator supports it too ofc). There are some issues with Sixel that I hope will be solved in the next release, until then they can usually be fixed by clearing the Zellij cache (~/.cache/zellij by default on linux) and restarting.

[bentenjamin]

A side effect of this is nushell is unusable in zellij when its use_kitty_protocol config option is true, because whatever escape sequence is sent causes the output to be cleared, meaning you get zero output from any command

[bew]

A side effect of this is nushell is unusable in zellij when its use_kitty_protocol config option is true, because whatever escape sequence is sent causes the output to be cleared, meaning you get zero output from any command

AFAIK the kitty graphics and keyboard protocol are separate and don't require each other, are you sure your comment is on the right issue?

[Straffern]

Is this an issue of either move to WezTerm or change to Tmux?
I as a Kitty user ran into this issue, when I tried to configure Image.nvim, and realizing other users had same problem with Zellij.

[merikan]

I can confirm that images does not display when using Ghostty Terminal. Ghostty does support the Kitty graphics protocol and images are displayed in Yazi when not in Zellij.

MacOS Sonoma 14.6.1 arm64
Zellij 0.41.2
Ghostty 1.0.0
Yazi 0.4.2

[vishalpaudel]

Ghostty+yazi works, Ghostty+zellij+yazi doesn't work.

Ghostty 1.0.1-arch
Yazi 0.4.2 (Arch Linux 2025-01-12)
zellij 0.41.2