zellij
zellij copied to clipboard
zellij-org
[Feature Request] - Implement kittys terminal graphics protocol
It would be neat if zellij could use kitty terminal graphics to display images or other files. Tmux had a similar issue closed, it seems like they will not be implementing it. Zellij does provide support for sixels but this is a very old protocol with not much support from other applications.
FIY: On the yazi wiki, there's a fix to use it with tmux: https://github.com/sxyazi/yazi/wiki/Image-preview-within-tmux This looks like a workaround, not sure how it works exactly but maybe zellij can do something similar?
FIY: On the yazi wiki, there's a fix to use it with tmux: https://github.com/sxyazi/yazi/wiki/Image-preview-within-tmux This looks like a workaround, not sure how it works exactly but maybe zellij can do something similar?
Yes, but it's extremely slow on tmux for some reason.
Is there a sense of the work required to implement this?
There's a plethora of users that want to enjoy images in their terminal and some terminal tools implement image support (image.nvim plugin for neovim, neorg plugin for neovim, a lot of file browsers, etc.). I almost never leave terminal and currently use tmux, image implementation in zellij would be a dealbreaker for me.
I really want to see Zellij support kitty graphics protocol, which is currently the most complete and advanced.
It supports Unicode placeholders, I think this could solve the stuttering issue in Zellij's Sixel implementation. Kitty protocol simply prints some escape sequence to the screen, and terminals that support that protocol display it as an image and accurately render the image position when scrolling.
This process is transparent to Zellij, and it doesn't need to parse and understand the meaning of the protocol itself, making it efficient.
I've just implemented Unicode placeholders for Yazi and tested it in kitty+tmux, it works well:
https://github.com/zellij-org/zellij/assets/17523360/7a40ed76-e406-4615-8029-6d3c7d636926
@sxyazi Is there something that needs to be done on zellij's side for this to work? I'm testing yazi (with the latest commit) from inside zellij, but image previews still doesn't work.
I've also tested it under tmux and it kinda works, but everything is really buggy (overlapping images, preview not updating, etc.) for some reason.
Is there something that needs to be done on zellij's side for this to work? I'm testing yazi (with the latest commit) from inside zellij, but image previews still doesn't work.
Yes, Zellij does not support it; this requires Zellij to provide support.
I've also tested it under tmux and it kinda works, but everything is really buggy (overlapping images, preview not updating, etc.) for some reason.
What are your Yazi version and tmux version? What terminal are you using? Please also do an echo $TERM, $TERM_PROGRAM, $XDG_SESSION_TYPE and paste the output here.
Just came across this and thought I should chime in.
I believe what's required on zellij's end is a Passthrough sequence (I don't know if such exists yet but I suspect not) i.e some something like this. No special handling of graphics control sequences should be required.
This should be all that's required to get Kitty's virtual placements working, assuming Zellij already supports indexed- (and optionally, direct-) -color SGR sequences and passes them through as recieved.
It'll also allow the use of other image/graphics protocols to a certain extent, particularly in full-screen (non-scrolling) applications. For full/proper support, specialized implementations/handling will be required but personally, I'd discourage that for a terminal multiplexer.
This is also how this issue was solved with junegunn/fzf junegunn/fzf kitty image protocol passthrough fix
He uses a regex to detect the terminal sequence for images then assigns a no-op function for the preview that allows the un-altered terminal sequence to passthrough to the terminal so that the image is rendered like normal.
You can test this with:
fd . ~/Pictures -e png | fzf --preview='kitty icat --clear --transfer-mode=memory --stdin=no --place=${FZF_PREVIEW_COLUMNS}x${FZF_PREVIEW_LINES}@0x0 {}'
I don’t think anyone has started to implement this. Just been talking about ways in which it could be achieved
I've been digging into this a bit more since I would like to see it implemented. It seems that zellij uses alacritty's vte parser to interpret ANSI/VT codes in terminal_pane.rs. Perhaps in the handle_pty_bytes function we can do a similar approach to what was done for fzf and do a no op and not pass in those bytes to the vte_parser
Fwiw I've tried the following:
- Kitty + yazi :heavy_check_mark:
- Kitty + tmux + yazi :heavy_check_mark:
- Kitty + Zellij + yazi :x:
- Wezterm + yazi :heavy_check_mark:
- Wezterm + tmux + yazi :heavy_check_mark:
- Wezterm + Zellij + yazi :heavy_check_mark:
❯ neofetch
██████████████████ ████████ jimc@kapua
██████████████████ ████████ ----------
██████████████████ ████████ OS: Manjaro Linux x86_64
██████████████████ ████████ Host: NH5x_NH7xHP
████████ ████████ Kernel: 6.1.99-1-MANJARO
████████ ████████ ████████ Uptime: 10 days, 15 hours, 22 mins
████████ ████████ ████████ Packages: 3373 (pacman), 75 (nix-user), 48 (nix-default)
████████ ████████ ████████ Shell: zsh 5.9
████████ ████████ ████████ Resolution: 1920x1080
████████ ████████ ████████ DE: GNOME 46.3.1
████████ ████████ ████████ WM: Mutter
████████ ████████ ████████ WM Theme: Custom-Accent-Colors
████████ ████████ ████████ Theme: Adw-dark [GTK2/3]
████████ ████████ ████████ Icons: Papirus-Dark-Maia [GTK2/3]
Terminal: kitty
CPU: 11th Gen Intel i7-11800H (16) @ 4.600GHz
GPU: Intel TigerLake-H GT1 [UHD Graphics]
GPU: NVIDIA GeForce RTX 3060 Mobile / Max-Q
Memory: 41611MiB / 64087MiB
@quadespresso That's great news, I didn't know it worked in Wezterm. Thank you for looking into this 👍
I've been using Alacritty for years and really missed the support for displaying images in the terminal and also ligatures. A few weeks ago decided to switch to Kitty and was somewhat surprised that images were not rendered in Zellij. After reading your comment I switched to Wezterm and everything works perfectly. 🥰
A funny thing was that when I, in Wezterm, attached to a running session started with Kitty, the images were not displayed. I had to close the session and restart it.
$ fastfetch
OS: macOS Sonoma 14.5 arm64
Host: MacBook Pro (16-inch, 2023)
Kernel: Darwin 23.5.0
Uptime: 22 hours, 26 mins
Packages: 42 (nix-default), 412 (brew), 133 (brew-cask)
Shell: zsh 5.9
Display (DELL U3419W): 3440x1440 @ 60 Hz in 34″ [External] *
Display (Color LCD): 3456x2234 @ 120 Hz (as 1728x1117) in 16″ [Built-in]
DE: Aqua
WM: Quartz Compositor
WM Theme: Multicolor (Dark)
Font: .AppleSystemUIFont [System], Helvetica [User]
Cursor: Fill - Black, Outline - White (32px)
Terminal: zellij 0.40.1
CPU: Apple M2 Max (12) @ 3,50 GHz
GPU: Apple M2 Max (30) @ 1,40 GHz [Integrated]
Memory: 33,45 GiB / 64,00 GiB (52%)
Swap: Disabled
Disk (/): 649,98 GiB / 926,35 GiB (70%) - apfs [Read-only]
Battery: 100% [AC connected]
Power Adapter: 90W
Locale: en_US.UTF-8
@merikan interesting test scenario.
Based on that, I tried the opposite: I ran wezterm -> zellij -> yazi then navigated to an image to view.
Then I detached from the zellij session, reconnecting from a kitty terminal. It was a bit of a mess, an entire screenful of characters. Nothing vaguely resembling yazi was to be seen.
@quadespresso That was interesting. I had to try it too and can confirm that it just gets a bunch of characters all over the screen. A little surprising that it is not only the image on the right side that gets distorted, but also the directory structures etc.
I couldn't resist, but also had Alacritty connected to the same session and there everything looked ok, except that the image is not displayed as we already know.
Kind of fun to look at multiple clients connected to the same session when jumping around in Yazi 😁
@merikan it's always interesting to cross-check with alternatives. I ended up trying the same combo with Alacritty and saw the same, ie, yazi doesn't display the image, but is otherwise intact.
Had a quick go with a few other terminal emulators (Terminator, Terminology, GNOME Terminal, Tilix), but only got as far as doing a quick check with yazi (minus zellij) to verify it couldn't display images before proceeding beyond that. I couldn't recall whether any of those supported inline images, and figured this would be quicker than pulling up docs for each.
@quadespresso Which version of "Wezterm + Zellij + yazi" that can display image? My version doesn't work:
- WezTerm 20240203-110809-5046fc22
- Zellij 0.40.1
- Yazi 0.3.1
None. Zellij would have to implement some form of a passthrough ansi sequence that wraps image protocol sequences that indicate that the sequence should be let through the multiplexer. tmux/wiki/FAQ#what-is-the-passthrough-escape-sequence-and-how-do-i-use-it Im pretty sure zellij just strips the escape code which leaves a mess of base64 image data
currently icat for the Image kitty protocol automatically detects if its inside of tmux and uses unicode placeholder data and the special tmux ansi escape to wrap its encoded image data.
@hongquan here we go:
- WezTerm 20240203.110809.5046fc22-2
- Zellij 0.40.1-1
- Yazi 0.2.5-2
What's odd is that as I tested this today, I got nothing when I navigated yazi to the image. Blank in the area where I expected to see the image,
I then quit yazi, quit zellij, started yazi, verified that I could see the image, quit yazi, started zellij and then yazi, and then I could see the image without issue.