lz-string
lz-string copied to clipboard
Implement IRI Encoding
Excuse my poor understanding of this topic in advance.
Something that occurred to me is that it should be possible to exceed the compression we have via compressToEncodedURIComponent()
using IRIs. The logic being simply that URIs only allowed for ASCII characters, but IRIs allow for the millions of different unicode characters, so by using these extra characters we should be able to encode some input more efficiently. Notably the goal for this kind of thing is to make the shortest URL, which isn't necessarily the same as making the smallest memory representation (because Unicode characters are larger than ASCII ones in memory). Also, this is different to the existing compressToUTF16
function, because not all unicode characters are legal in IRIs, although most are.
If this isn't particularly difficult, I'm happy to contribute this functionality if I'm able. Having looked at the existing implementation, I wonder if this is as simple as providing a new "conversion" function that encodes integers into strings. This could be done similarly to how you have done it with URIs: https://github.com/pieroxy/lz-string/blob/2f749bc9fe3cc889fa9e32d2769a930d977596e6/libs/lz-string.js#L95
For reference, the IRI standard that defines the available characters is here: https://datatracker.ietf.org/doc/html/rfc3987#section-2.1. The part that is of relevance is the iquery
rule, which defines the legal IRI query string:
ipchar = iunreserved / pct-encoded / sub-delims / ":"
/ "@"
iquery = *( ipchar / iprivate / "/" / "?" )
ifragment = *( ipchar / "/" / "?" )
iunreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" / ucschar
ucschar = %xA0-D7FF / %xF900-FDCF / %xFDF0-FFEF
/ %x10000-1FFFD / %x20000-2FFFD / %x30000-3FFFD
/ %x40000-4FFFD / %x50000-5FFFD / %x60000-6FFFD
/ %x70000-7FFFD / %x80000-8FFFD / %x90000-9FFFD
/ %xA0000-AFFFD / %xB0000-BFFFD / %xC0000-CFFFD
/ %xD0000-DFFFD / %xE1000-EFFFD
iprivate = %xE000-F8FF / %xF0000-FFFFD / %x100000-10FFFD