contour
contour copied to clipboard
contour-terminal
SSE/AVX based Base64 decoder
Just like UTF-8 decoding, Having an SSE/AVX based Base64 decoder will definitely improve performance on loading Sixel images or any arbitrary payload coming in via the VT stream.
There is a very nice article on that at http://0x80.pl/notesen/2016-01-17-sse-base64-decoding.html which also points to https://github.com/WojciechMula/base64simd/ as a demo implementation.
Once ready, we can add https://github.com/contour-terminal/base64-cpp via CPMAddPackage() to contour and just kill the local base64 code and its tests. With that repo, I want to provide a general purpose base64 easy-to-use encoding/decoding API that makes also use of SIMD, as based on WojciechMula's work.
Could you offer that under MIT? That would allow more OSS projects like Windows Terminal to use it if it is as helpful to use to them as it is to you. (Though since MS has released stuff under Apache, that license may be ok for companies like them to consume).
I have learned that there is a general bad experience with MIT license in well known open source projects. Let's see. Since the main work will be dual license, we MAY want to chose dual license too. But only if there is a possibility to get it into other prominent projects that otherwise would not be able to. (WT e.g.)
They've accepted the Boost license and you'll need to have a ready solution completed before we could raise their attention to it. But maybe there's another sponsor you could use. I probably wouldn't worry about the dual license just yet. Apache is probably fine and if that changes, you can look at it then.
I am currently focusing on getting UTF-8 decoding performance increased via SIMD (in libunicode). When this is done and merged, I'll resume base64 library to be more complete.
@christianparpart Also interested in your solution here (prolly would use a variation as a wasm build) :smile_cat:
Regarding UTF8:
For ASCII-only data (like DCS stuff, even better if you could skip it for that) I found Bob Steagall's zero padding with SIMD most useful perfwise (can be made even faster with SSE4.1 with _mm_insert_epi32 + _mm_cvtepu8_epi32). As he points out in his video, a SIMD utf8 decoder always will have a tradeoff between ASCII vs. n-byte chars. For terminal data we know, that it is >95% ASCII data most of the time, thus I suggest to optimize for that in the first place, and the increasing n-byte cases in decreasing priority (ASCII > 2 > 3 > 4, with 4-byte chars being very rare).
More links: ;) https://github.com/lemire/fastbase64 https://github.com/powturbo/Turbo-Base64 https://github.com/aklomp/base64 https://github.com/npodonnell/fast-base64
