clockworklabs
Provide option for gzip
Puru:
Provide an option in WS API to send the response encoded in gzip via
?gzipor?encoding=gzipOR
Totally switch to gzip, which should be a non-breaking change(AFAIK).
From discord:
"Just outlining the general outline: If we wait for brotli in web, and it comes after 1 year, it'll take over 1 more to > spread across all the browsers(* cough * Safari firefox * cough *) and have > enuf usage as well(Like safari being tied to mac updates so it'll be the slowest to get). After that, it would take time for Node to get it as well, after which we gotta wait for us bumping our TS SDK's minimum node version to the one which supports DecompressionStream(brotli) So we're looking at 2-3 years from now at a minimum. Which means 100KB for our SDK for the said period of time
If we switch to gzip or provide it as an option, we can have a non-breaking > change in the JS SDK(which internally starts using DecompressionStream('gzip') widely supported). We can also do this as a breaking change while bumping up a minimum node version, pre-1.0. And the bundle size can go down very easily to 5-10KBs, which is really good to have. Sizes don't matter in the long run, but it does if people start complaining and this causes others to try to use HTTP API instead, which is a failure of the SDK.
Something to think about pre-1.0"
Isn’t gzip an implementation detail for the sdks? If so, it shouldn’t break anything
Unless people are manually using the binary version of ws api, in which case it would be breaking.
Further confirmation required from Phoebe/Mazdak
Breaking in the sense that:
- We will need to update the SDKs.
- SDK users will need to update to a new SDK version.
- Non-SDK users of the binary WebSocket API (afaik none exist) will have to change their code.
Not breaking in the sense that SDK users would need to change their code.
Since Joshua has confirmed that GZip performs similarly to Brotli for our needs, assuming we can locate reliable Rust and C# (.NET and Unity) libraries for it, this seems like a totally reasonable and low-effort thing to do so long as we do it pre-1.0 so we don't break a stable API. I make no comment as to how high priority this should be.
FWIW we considered this a while ago, and ended up deciding that we should first do https://github.com/clockworklabs/SpacetimeDB/issues/1080 before switching the compression algorithm, since we believe that that might address most of the performance issue.
(Edit: ah I didn't read the motivation that apparently web doesn't support brotli?)
apparently web doesn't support brotli?
Manual brotli decompression isn't supported yet 😄
We've decided to support both brotli and gzip, rather than replace.
How are we planning to support this? Would this be a non-breaking change (e.g. just allow you to provide an option and use Brotli by default) or change the default or require that you specify which compression scheme you want.