Support brotli encoding in HAR capture

[mike10004]

When capturing responses for a HAR, response content that is brotli-compressed (identified by a Content-Encoding header with value "br") is not decompressed. This results in the HAR containing base64-encoded content for those responses where it would be helpful to have plain text. Scripts and stylesheets are frequently served with brotli compression.

Currently, the ServerResponseCaptureFilter class only supports decompressing gzip-compressed response content in the decompressContents() method:

protected void decompressContents() {
    if (contentEncoding.equals(HttpHeaders.Values.GZIP)) {
        try {
            fullResponseContents = BrowserMobHttpUtil.decompressContents(getRawResponseContents());
            decompressionSuccessful = true;
        } catch (RuntimeException e) {
            log.warn("Failed to decompress response with encoding type " + contentEncoding + " when decoding request from " + originalRequest.getUri(), e);
        }
    } else {
        log.warn("Cannot decode unsupported content encoding type {}", contentEncoding);
    }
}

It would be helpful to support decompression of brotli-compressed content in this class, and moreover, it would be helpful to support creation of a custom ServerResponseCaptureFilter in the HarCaptureFilter constructor, perhaps by providing a factory as a constructor parameter.

There are some other technically valid content encodings that are not supported by the current server response filter; for example, the Content-Encoding header might be "identity" or might specify multiple encodings. The factory approach would allow Browsermob users to implement their own response content manipulations as needed.

The library org.brotli:dec:0.1.1 provides a pure-Java brotli decompression implementation that could be used for this purpose. I'd be glad to try out a patch to support brotli and submit a pull request for the functionality suggested here.

(Tested with version 2.1.5.)

[n0m0r3pa1n]

Hi @mike10004 , I saw this right ticket a minute ago. I saw the brotli problem because I wanted to get some Facebook data and I did this patch: https://github.com/lightbody/browsermob-proxy/pull/742 :+1: You can try if it works for you, works fine on my side :)

[mike10004]

Thanks! That looks like a good implementation -- I'm just waiting to see what the browsermob-proxy developer says about the factory idea.

[jrgp]

Hi folks, any traction/update on this? I think that PR looks good. 👍

[ericbeland]

It seems that development on the browsermob proxy has ceased. We have a fork, the browserup proxy that has merged brotli support. It should be a drop in replacement. https://github.com/browserup/browserup-proxy