GZip on Azure Functions V3 Not Working For JSON Response

[unearthly85]

trafficstars

Hey everyone,

"I feel like I'm taking crazy pills", but I can't find the answer to my problem... I have a group of Azure Functions V3 that are serving up a REST API. They are on an "Always On" App service plan of B1.

In front of our Azure Functions, we are using Azure API Management. (Although this problem occurs when we make calls directly to the functions)

We have a larger API response that returns about 375k of JSON, and noticed that it wasn't being compressed (GZIP).

Documents online seems to say it should just work by default, but mine isn't:

https://stackoverflow.com/questions/49339268/how-can-i-enable-gzip-compression-on-azure-functions-v2

Also other people seemed to have this issue previously: https://github.com/Azure/azure-functions-host/issues/4000

I have been searching online for hours, and I have even searched for ways to disable gzip, so I could find the location of the settings so I could verify it wasn't disabled, but to no avail.

I even thought maybe it's compressed but somehow I'm missing it, so I tested it on webpagetest.org:

Related question post: https://docs.microsoft.com/en-us/answers/questions/353996/gzip-on-azure-functions.html

[v-anvari]

Hi @unearthly85 , Thank you for reaching out to us, checking internally on this issue

[v-anvari]

Hi @unearthly85 , Are you using Windows or Linux? If it is Linux, can you check if you are using CORS or EasyAuth (Authentication) features, as that puts in a layer of middleware that may not work well with GZIP compression.

[unearthly85]

Hey @v-anvari ! We are using linux. We are not using CORS or any Authentication features (Authentication has no identity providers, and Authentication Classic is off).

[v-anvari]

Hi @unearthly85 , Thank you for the update, we are checking internally for any known issue

[v-anvari]

Tagging @ConnorMcMahon, for more insights

[v-anvari]

Hi @unearthly85, We reproduce that the gzip compression just works on Windows, and doesn't work on Linux. Checking this internally if this is explicitly supported in Linux App services or this support needs to be included. Meanwhile, Please go through the following link to check if the steps work for your project scenario how-do-i-enable-gzip-compression-for-linux-azure-app-services

[unearthly85]

Hey @v-anvari, When testing locally, there is a problem I find when manually adding the header for Content-Encoding to my .net azure functions.

I attempted to do what was mentioned in how-do-i-enable-gzip-compression-for-linux-azure-app-services.

So we just need to add Content-Encoding: gzip in Response Headers, then we can solve the issue.

Tested by adding the following line: req.HttpContext.Response.Headers.Add("Content-Encoding", "gzip"); to a single response produces the following error in postman:

I tested this before posting this issue with no results. Is it possible this is because I'm testing locally? I haven't tried this deployed.

[unearthly85]

I am looking into builder.Services.AddResponseCompression(); suggested here: https://stackoverflow.com/questions/61688759/enabling-gzip-compression-on-azure-app-service-for-containers

This doesn't work on localhost, but supposedly this isn't possible as seen in the link below, and it does not create any errors on response like manually adding the Content-Encoding: gzip header. https://stackoverflow.com/questions/49339268/how-can-i-enable-gzip-compression-on-azure-functions-v2

I will see if this works when deployed.

[unearthly85]

@v-anvari , It appears that Azure Functions doesn't support middleware the same way that Web API does, so I cannot implement gzip response compressions in code like suggested in my prior comment.

At this point, I do not know any way to work around this issue and I am stuck with linux app services being unable to use gzip compression, so I have deployed a new Function app on windows infrastructure.

Edit: Worth noting, windows based app service plans for the same performance are almost 4x more expensive.

• B1 Windows: $54
• B1 Linux: $14

Yeah, not great :-/

[v-anvari]

Hi @unearthly85 , Thank you for sharing your findings. We are checking with the right team to confirm the support on this scenario.

Cc @ConnorMcMahon, for confirming the supportability

[danielgary]

This appears to still be broken. I've worked around it by using zlib, but performance takes a hit.

[mithunshanbhag]

@unearthly85: A possible workaround (note: I haven't tried this out myself).

If you're using API management (gateway) to call into Azure functions, then you can set a policy to force gzip compression.

https://stackoverflow.com/a/60312137

https://github.com/jiasli/azure-notes/blob/master/api-management/sample-policies.md#gzip-compression

[dixonwille]

I am seeing the same thing with Azure Functions 4

1. Linux App Service
2. Containers
3. Can't set header manually as local breaks

Is there any work being done on this? I really don't want to deploy to Windows as we just got off of it because of pricing. Seems silly to go back to it just for compression. I was thinking that maybe the container didn't have gzip library installed, but it does.

[mindraptor]

Any update on this topic? Also looking forward to this feature for linux plan :)

[wojtekmaj]

Blows my mind that I need to deal with this on my own, but here's my solution (Brotli only since it's effective and universally accepted, but easy to expand if needed).

import brotli from 'brotli';
import type { HttpRequest, HttpResponse, HttpResponseInit } from '@azure/functions';

// deflate, gzip;q=0.8, br;q=1.0 -> ['br', 'gzip', 'deflate']
function parseAcceptEncoding(acceptEncoding: string | null): string[] {
  if (!acceptEncoding) {
    return [];
  }

  const encodings = acceptEncoding
    .split(/,\s*/)
    .map((encoding) => {
      const [name, q] = encoding.split(';q=');

      if (!name) {
        throw new Error('Invalid accept-encoding');
      }

      return { name, q: q ? parseFloat(q) : 1 };
    })
    .sort((a, b) => b.q - a.q)
    .map(({ name }) => name);

  return encodings;
}

function getResponse(req: HttpRequest, res: HttpResponseInit): HttpResponseInit {
  const acceptEncoding = req.headers.get('accept-encoding');

  const encodings = parseAcceptEncoding(acceptEncoding);

  const acceptsBrotli = encodings.includes('br');

  if (acceptsBrotli) {
    if (res.body && (typeof res.body === 'string' || res.body instanceof Buffer)) {
      const { body, headers, ...rest } = res;

      return {
        ...rest,
        headers: {
          ...headers,
          'content-encoding': 'br',
        },
        body: brotli.compress(typeof body === 'string' ? Buffer.from(body) : body),
      };
    }

    if (res.jsonBody) {
      const { jsonBody, headers, ...rest } = res;

      return {
        ...rest,
        headers: {
          ...headers,
          'content-encoding': 'br',
          'content-type': 'application/json',
        },
        body: brotli.compress(Buffer.from(JSON.stringify(jsonBody))),
      };
    }
  }

  return res;
}

[cajuncoding]

Still seems to be an issue in our Linux deployed Azure Functions as there is still no automatic handling of response compression based Accept-Encoding header.

However, it's not too hard to implement manually in .NET and I provide a fully functioning version that supports Gzip, Brotli, and Deflate in a gist here: https://gist.github.com/cajuncoding/a4a7b590986fd848b5040da83979796c

Can be implemented as:

[FunctionName(nameof(MyHttpTriggerWithResponseCompressionSuppport))]
public Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = "compressed-response-api")] HttpRequest req, ILogger logger)
{
    req.EnableResponseCompression(); //<== Enable Response Compression BEFORE writing Anything to the response!

    var httpResponse = req.HttpContext.Response;
    httpResponse.Headers.ContentType = "application/json";

    //DO some work and write to the HttpContext Response Body stream... all content will be compressed...
    await using var responseWriter = new StreamWriter(httpResponse.Body, Encoding.UTF8, leaveOpen: true);
    await responseWriter.WriteAsync(JsonSerializer.Serialize<object>(new
    {
        Topic = "Hello World...I'm Compressed!",
        Body = "If I was some big JSON then I'd be nicely packed for transmission!"
    }));

    return new EmptyResult();
}

For posterity I'm also reposting the source here, but going forward any fixes/enhancements will be in the Gist link above...

using System.IO.Compression;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Primitives;

namespace AzureFunctionsInProcessCompressionExtensions
{
    /// <summary>
    /// BBernard / CajunCoding (MIT License)
    /// Extension method for the HttpContext & HttpRequest to enable response compression within Azure Functions using the In Process Model -- which
    ///     does not support custom Middleware so it cannot easily intercept & handle all requests.
    /// There are many issues reported with response compression not always working, particularly with Linux deployments,
    ///     therefore this helps to easily enable this for any request by simply calling the extension method in the Function invocation.
    /// It works by simply inspecting the AcceptEncoding header to determine which, if any, compression encodings are supported (Gzip, Brotli, Deflate)
    ///     and then wrapping the Response Body Stream with the correct implementation to encode the response while also setting the correct Response Header
    ///     for the Client to correctly decode the response.
    ///
    /// This works great with GraphQL via Azure Functions (In Process Model) using HotChocolate GraphQL server allowing for compression of large
    ///     GraphQL Query results which can significantly improve performance in use cases with large query result sets.
    /// </summary>
    internal static class AzureFunctionsInProcessCompressionExtensions
    {        
        public static class AcceptCompressionNames
        {
            public const string Gzip = "gzip";
            public const string Brotli = "br";
            public const string Deflate = "deflate";
        }

        public static HttpRequest EnableResponseCompression(this HttpRequest httpRequest)
            => httpRequest.HttpContext.EnableResponseCompression().Request;
        public static HttpContext EnableResponseCompression(this HttpContext httpContext, CompressionLevel compressionLevel = CompressionLevel.Fastest)
        {
            var acceptCompressionTypes = httpContext.Request.Headers.AcceptEncoding
                .SelectMany(v => v.Split(','))
                .Select(v => v.Trim().ToLower())
                .ToArray();

            if (acceptCompressionTypes.Length <= 0)
                return httpContext;

            var httpResponse = httpContext.Response;
            var responseStream = httpResponse.Body;
            var responseHeaders = httpResponse.Headers;

            foreach (var compressionType in acceptCompressionTypes)
            {
                switch (compressionType)
                {
                    case AcceptCompressionNames.Gzip:
                        httpResponse.Body = new GZipStream(responseStream, compressionLevel, leaveOpen: false);
                        responseHeaders.ContentEncoding = new StringValues(AcceptCompressionNames.Gzip);
                        return httpContext;
                    case AcceptCompressionNames.Brotli:
                        httpResponse.Body = new BrotliStream(responseStream, compressionLevel, leaveOpen: false);
                        responseHeaders.ContentEncoding = new StringValues(AcceptCompressionNames.Brotli);
                        return httpContext;
                    case AcceptCompressionNames.Deflate:
                        httpResponse.Body = new DeflateStream(responseStream, compressionLevel, leaveOpen: false);
                        responseHeaders.ContentEncoding = new StringValues(AcceptCompressionNames.Deflate);
                        return httpContext;
                }
            }

            return httpContext;
        }
    }
}