browsh Support libsixel for rendering?

Support libsixel for rendering?

Open vrthra opened this issue 5 years ago • 21 comments

Sixel is a way to do (non-pixelated) graphics and animation in the terminal such as Xterm. Support for its use is provided by libsixel. It can do animation or render images directly on the dec terminals. Here is netsurf (a tiny browser) running on an xterm. It would be great if browsh could support libsixel and produce non-pixelated images and animations on terminal.

Jul 10 '18 09:07 vrthra

Thanks for the suggestion but I'm actually already pretty decided against this. Simply because I want the basic Browsh engine to be as text-focused as possible, as then we have a universal source of the modern web in pure text format, that multiple other clients (terminals being only one) can make use of. And besides graphics are orders of magnitude more bandwidth heavy than text, which somewhat defeats the purpose of Browsh.

Jul 10 '18 10:07 tombh

How about injecting some JS into webpages to allow opening images in a popup with more "pixels"?

https://s33.postimg.cc/ab7p4xwan/Selection_094.png

https://gist.github.com/certik/4336299de10f400ee49943bd9f8a8ba6

Jul 10 '18 10:07 osowiecki

@tombh: I think there's something to be said for a hybrid mode. Text-only should be the default, but the user should have the ability to request the full image. Both in tty and http mode (or a future dedicated client/protocol). At the very least the user should be able to retrieve the image's URL.

I think @osowiecki's suggestion to use JS to make it possible to load the image in a popup is great for http server mode.

Jul 10 '18 11:07 tobimensch

Just being able to open the image in new tab should do the trick. Adding support for opening links in new tab would solve 2 issues at once.

Jul 10 '18 11:07 osowiecki

@tombh

And besides graphics are orders of magnitude more bandwidth heavy than text, which somewhat defeats the purpose of Browsh.

I did not understand this. You are any way fetching images and videos to show it with aalib right? How does showing graphics on the xterm with libsixel instead of aalib have an impact on the bandwidth?

Jul 10 '18 11:07 vrthra

The idea is (as I understand it) that your remote server downloads entire image and then only sends a pixelated representation to your machine over internet (ssh).

Jul 10 '18 11:07 osowiecki

@osowiecki: Correct.

The use case of replacing elinks/lynx/w3m because browsh is a better text browser and not for bandwidth reasons wasn't the primary use case in mind when developing browsh.

So serving pixel perfect images hasn't been implemented because it doesn't help people stuck on very bad internet connections much.

On top of that there's no definite standard for displaying images in terminals, for example terminology has custom support for it, and then there's libsixel and whatnot, but there's no common standard, which means this is a lot of busy work.

I think @tombh might accept patches for features such as that, if someone wants to do work.

Jul 10 '18 11:07 tobimensch

By the way. Pixelated images prevent people from solving google captchas (find all them cars). Maybe you can render those in better quality?

Jul 10 '18 12:07 osowiecki

@osowiecki: That's true. There are many applications that are next to impossible with pixelated images. Which means that browsh in its current form is simply not a general purpose browser. I hope that it can eventually get there, but first it must master it's speciality, which is low bandwidth text browsing, really well.

Jul 10 '18 12:07 tobimensch

Some good ideas, thanks. So how about this:

In the TTY client have some way to special-click on an image so that it takes up the whole TTY, whether that's in a new tab or the current tab. Bear in mind that it is impossible to increase the resolution of a TTY without the user manually decreasing the font size.
(1) will be additional to the magnifying glass: #33
In the HTML service, inject all <img> elements with the original image's src so that again you can click if you feel inclined to download the full image.

Solving captchas is the perfect use case for the magnifying glass. Thoughts?

Jul 13 '18 08:07 tombh

Closing for #137 and #147.

Jul 20 '18 06:07 tombh

I don't think the developer understands how sixels actually work, you have to understand that images are not being sent, instead libsixel is converting the image to a specific text sequence that is then rendered as an image on the client terminal, the overhead is minimal and would allow full image support from browsh. At least consider the option as browsing the web in purely text is a little too minimal in some cases, there are times where higher resolution images are required and sixel support can definitely help provide that while still keeping bandwidth usage very low.