joouha
partial sixel rendering
While playing with your euporie browser example I saw that you do a low quality prerendering of graphics while they are not fully in viewport, and once they are fully in the viewport, switch to better quality with sixel.
This currently raises 3 issues:
- low UX, as peeps first see a blurry preview like thingy
- issues with overly big graphics not fitting into the viewport - no chance to get nice quality output here at all?
- (blurry preview still has cell width issues, at least in the browser variant)
Thus I wonder if it would be feasible to do partial sixel rendering.
Now the main issue with sixels is - it is very expensive to convert the original image into sixel in the first place, as it needs quantization and dithering. So ideally one want to do that only once and cache the result. On the other hand the advanced image algos needed for sixel conversion kinda make pre-slicing the original image into line parts and separately convert to sixel a no-go, as the created color palettes and dithering pattern are not stable (would lead to ugly stitching artefacts).
A possible solution to all those downsides would be a sixel-cropping library, eg.:
- create full sixel repr of the original image
- crop into that sixel as needed (pull pixel area WxH with needed colors)
- print the cropped part
A more involved solution would be to write own sixel converter capable to freeze the color palette, and do the substitution on slices of the original image. This would need one initial quantization step creating a color substitution map with dithering marks. With that slices can be created pretty fast and memory efficient. (I suggested something similar here for the julia peeps, as they are keen to code such a thingy on their own).
So I wonder if you are interested in such an enhancement for sixel output. In general I'd lean more towards to the sixel-crop library idea, which should be straight forward to implement.
Hi again!
Partial image rendering is something I'm planning to implement. I think cropping the images in sixel-land is a good idea.
I had originally planned to just crop the images in PNG-land and re-convert them each time (but you're right, this would be slow), and I have tried slicing images and converting the parts separately (but you're right that it doesn't work well).
Images in euporie are always aligned to the top edge terminal cells, which makes things a bit simpler, although a sixel-cropping library would need to be able to edit the last row of sixels to blank out some pixel-rows. A more complex use-case would be aligning cropped sixel images to the bottom edge of a cell, as this would require shifting pixels between sixel boundaries.
I also support the kitty and iTerm2 graphics protocols in euporie, which would need to support partial image rendering too:
- Kitty's protocol allows you to specify the region of the image to display, so should be straight forward to implement;
- iTerm2's protocol basically consists of dumping base64 encoded images, so images would need to be cropped in PNG-land.
I'll need to refactor euporie's format conversion system to allow images to be cropped at arbitrary points in the conversion chain, then a sixel-cropping library would slot right in.
Is a sixel-cropping library something you're interested in writing?
Is a sixel-cropping library something you're interested in writing?
Will see what I can do about it. But I would code it in C most likely for speed reasons (and no not in rust as everyone else does nowadays :smile:). This way I can port it to wasm once I need it on xterm,js side (actually I already do for a pending feature of state serialization, which currently skips any graphics content).
A more complex use-case would be aligning cropped sixel images to the bottom edge of a cell, as this would require shifting pixels between sixel boundaries.
True, but I think a crop lib should be able to handle that, ideally with totally independent output positioning. Imho it should be possible to select some rectangle on the original sixel and copy it over to a totally different position on the target. The skipped over pixels could simply inherit the background_select param of the original sixel image (or even make that customizable, wotever fits the purpose).
I've started working on implementing sixel cropping in python. I've still got some bugs relating to cropping in the x-direction to iron out, but it the concept works!
@joouha I have currently not much time to work on this, so if you have a working python lib for it - just use that for now. I'll get back to this once I fixed some other fundamental issues I still have with my sixel lib (https://github.com/jerch/node-sixel/issues/58).
Here's my sixel cropping Python library: https://github.com/joouha/sixelcrop
I'm planning to implement this in euporie soon
This is now implemented in the dev branch, and will be in the next release 🪄
It works with all three terminal graphics protocols (sixel, kitty, and iterm)
https://user-images.githubusercontent.com/12154190/207418617-29140b2b-2203-4bf6-b143-21b232e6ea57.mp4