Add CompressionStream

[Symbitic]

What is the problem this feature would solve?

The compress middleware in Hono depends on CompressStream, which isn't implemented by WebKit. By adding it, it would increase the performance of transferring static assets such as CSS and SVG files.

What is the feature you are proposing to solve the problem?

Implement CompressionStream in Bun. That would automatically allow the compress middleware in Hono to work, and it would enable more Bun-optimized frameworks like Elysia to design their own middleware.

What alternatives have you considered?

It might be possible to implement CompressStream in WebKit and use that.

[Jarred-Sumner]

Yeah we need to do this

WebKit has a CompressionStream/DecompressionStream we could use and uWebSockets (our HTTP server lib) also has builtin support for compression. Neither are wired up yet.

[Jarred-Sumner]

Prerequisite 3b259211df10e06ddf5844898a6ffa3c9679e4d8

[vesamet]

Out of curiosity, what is the state of this issue? (My app really depends on Hono's compression middleware 😅)

[cloudspeech]

My node app uses DecompressionStream to unpack npm packages 'manually', i.e. without official npm tooling - I would very much like to switch to bun, but this is blocking me ATM.

[marc-barry]

We just tried to port our proxy from Deno to Bun as sadly Bun doesn't support compression on its own. So we have to use https://hono.dev/middleware/builtin/compress with Bun. Is there another way to handle HTTP compression in Bun?

[niklasgrewe]

any news about this issue? I also use Hono and want to use there compress middleware

[venkatd]

Our backend is unable to run without DecompressionStream, putting in my vote for this!

[tomek-f]

Any news about this issue?

[ewrogers]

Out of curiosity, what is the state of this issue? (My app really depends on Hono's compression middleware 😅)

+1 for the same reason, though I can disable it for now it would be great to have this soon.

[cloudspeech]

One could try https://github.com/101arrowz/compression-streams-polyfill in the meantime.

[jimmywarting]

I wired up my own compression stream polyfill that is based upon zlip instead of using fflate

// @bun

/*! MIT License. Jimmy Wärting <https://jimmy.warting.se/opensource> */
import zlib from 'node:zlib'

// fyi, Byte streams aren't really implemented anywhere yet
// It only exist as a issue: https://github.com/WICG/compression/issues/31

const make = (ctx, handle) => Object.assign(ctx, {
  writable: new WritableStream({
    write: chunk => handle.write(chunk),
    close: () => handle.end()
  }),
  readable: new ReadableStream({
    type: 'bytes',
    start (ctrl) {
      handle.on('data', chunk => ctrl.enqueue(chunk))
      handle.once('end', () => ctrl.close())
    }
  })
})

globalThis.CompressionStream ??= class CompressionStream {
  constructor(format) {
    make(this, format === 'deflate' ? zlib.createDeflate() :
    format === 'gzip' ? zlib.createGzip() : zlib.createDeflateRaw())
  }
}

globalThis.DecompressionStream ??= class DecompressionStream {
  constructor(format) {
    make(this, format === 'deflate' ? zlib.createInflate() :
    format === 'gzip' ? zlib.createGunzip() :
    zlib.createInflateRaw())
  }
}

( this is only needed in Bun - all other env have compression stream right now... )

[javalsai]

I think this might be relevant in here, I was doing stuff with gzip streams in bun and I found a weird bug with following code:

import { createGzip } from 'node:zlib'
import { Readable, pipeline } from 'node:stream'

const gzip = createGzip()

pipeline(
    Readable.toWeb(process.stdin),
    gzip,
    process.stdout,
    () => {},
)

Now, you should be able to run the program with echo <some data> | bun/node <path> > /dev/null and they should behave the same way, but in my case I was doing random data and big streams and found out from dd with bs=1K and count=16 up, the program hangs forever, so:

• dd if=/dev/urandom bs=1K count=15 | bun std-gzip-pipe.js > /dev/null: works in under 0.002 secs on my machine
• dd if=/dev/urandom bs=1K count=16 | bun std-gzip-pipe.js > /dev/null: hangs forever on bun, node works however

It turns out to be a stdio issue

[sbenzemame]

Any news about this issue?

[danthegoodman1]

This is a bit of a blocker for a use case I have for bun. In the meantime I can use third party packages but I hope this is supported soon.

[sunneydev]

Before this issue is resolved you can use https://www.npmjs.com/package/bun-compression. I just ported the compression middleware that Elysia was using for hono.

[codedread]

Does this bug also cover the DecompressionStream implementation? If so, can someone update the bug title?

[Mehuge]

My deduplication backup [ddb] software uses streams with compression, specifically it .pipe()s to and from a compressor (gzip/gunzip). It operates over http as well as over the file system. Loading an entire file into memory is ofc not an option, so has to use streams.

[danielniccoli]

It's on the roadmap https://github.com/oven-sh/bun/issues/159

I, too, hope for a soon implementation. 🙏☺️