Allowing content body over 100MB

[jabbera]

Is your question related to a specific version? If so, please specify: v3

What language does your question apply to? C#

Question

I'd like to know how to expand the content body for Http Triggers to support messages over 100MB. I have a historical analytics function I'd like to host in Azure Functions. This will be called from MATLAB which has pretty poor support for rest in general. I'd really like to avoid having to shuttle messages that serialize to over 100MB to blob storage. My Plan was to use EP1.

[jabbera]

@fabiocav here it is.

[mhoeger]

@fabiocav - it sounds like you might have some background on this. do you mind triaging?

[jabbera]

Origin of request to open new issue. https://github.com/Azure/azure-functions-host/issues/1063#issuecomment-608110162

[wlazuardi]

Ditto here! I'm having the same issue with size > 100MB

[jabbera]

Hi! Any progress on this?

[adadurov]

Ditto! Same issue here -- I sometimes need up to 400 MB. Would prefer not to rewrite/add other methods to my API.

[jabbera]

Hi all, any chance this has been solved?

[calloncampbell]

Is this resolved? Any reason why 100MB is set as a limit, especially across all SKUs? I'm using Premium Function and need support for like 500MB.

[Cmohan]

I am having this same issue. Almost all of Microsoft's services in Office and Azure seem to be artificially limited at 100MB. That make sense for consumer apps like Power Automate and Power Apps that are free, but it doesn't make sense for Azure apps like Functions or Logic Apps that are paid services. Is there any plan to allow larger requests or remove the limits entirely?

[daniel-packard]

I'm also having this issue. Looking for best guidance

@mhoeger - has this issue been triaged? @fabiocav?

https://github.com/Azure/azure-functions-host/issues/5854#issuecomment-609941455

@fabiocav - it sounds like you might have some background on this. do you mind triaging?

[daniel-packard]

PS: wondering if a pull request like this one would be welcome to increase the limit? Or is there a reason to keep it at a hard limit of 100MB for azure functions? Maybe expose a setting, but keep the default at 100MB?

https://github.com/Azure/azure-functions-host/pull/1271/files

[Exitare]

Having the same issue. Is there any update on this ?

[Pwd9000-ML]

Would be interesting to see if this limit does get lifted.

[jamesxv7]

I would love for this to become a reality. If it is not possible, it would be very good to understand the reason for this hard limit and possible solutions to deal with it other than depending on another API or another service.

[xyzpda]

I also want you to solve this problem.

[IanKemp]

Come ON guys, how is this not fixed yet? Or more to the point - WHY does this limit even exist to begin with in the first place?

[lloydjatkinson]

Just hit this crazy problem myself. Why 100MB?

[J-Cat]

I have no issue with 100MB as a default, however, why we can't increase that when running in our on premise environments or our development VSCode environments is beyond me!

Please Microsoft issue an update so it can configure these values through host.json

[J-Cat]

I am thinking that one way to solve this, at least if you are deploying to a custom Docker file, would be as part of the Docker image build, to overwrite the web.config settings in the Azure Functions WebJob Host. Doesn't help with local debugging, but does maybe get around increasing the limits in your actual deployment environments.

[RoygarBriceno-Arkano]

Hi, I'm facing the same issue with a project (and agree that this should be a simple setting to override) @J-Cat, your idea of using a docker image build worked?

[J-Cat]

Hi, I'm facing the same issue with a project (and agree that this should be a simple setting to override) @J-Cat, your idea of using a docker image build worked?

I actually had the settings wrong in my host.json! I forgot the "settings" key underneath "workflow" and had the actual settings just directly under it. The following works to override the settings you need:

{
  "version": "2.0",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle.Workflows",
    "version": "[1.*, 2.0.0)"
  },
  "extensions": {
    "workflow": {
      "settings": {
        "Runtime.Backend.HttpOperation.MaxContentSize": "314572800",
        "Runtime.ContentLink.MaximumContentSizeInBytes": "314572800",
        "Runtime.FlowRunActionJob.MaximumActionResultSize": "629145600",
        "Runtime.Backend.ApiConnectionOperation.MaxContentSize": "314572800",
        "Runtime.Backend.VariableOperation.MaximumVariableSize": "314572800",
        "Runtime.Backend.FunctionOperation.MaxContentSize": "314572800",
        "Runtime.Backend.HttpWebhookOperation.MaxContentSize": "314572800",
        "Runtime.Backend.HttpOperation.RequestTimeout": "00:10:00",
        "Runtime.Backend.HttpWebhookOperation.RequestTimeout": "00:30:00",
        "Runtime.Backend.HttpWebhookOperation.DefaultRetryCount": "0",
        "Runtime.Backend.HttpOperation.DefaultRetryCount": "0"
      }
    }
  },
  "logging": {
    "console": {
      "isEnabled": false,
      "DisableColors": false
    },
    "logLevel": {
      "default": "Error",
      "Function": "Error",
      "Host.Aggregator": "Error",
      "Host.Results": "Error",
      "Host.Executor": "Error"
    }
  }
}

[bluewilliams]

Has anyone had success overcoming the content size limit using the host.json technique in a V4 Runtime Azure function?

We tried with a verbatim copy of the above but still hit the request limit cap at 100 MiB. Possibly due to the extension version needing to be bumped?

i.e.

    "extensionBundle": {
        "id": "Microsoft.Azure.Functions.ExtensionBundle.Workflows",
        "version": "[4.0.0, 5.0.0)"
    },

[amycai-arcadis]

Is there any solution or progress on the above comments?

[jimdigriz]

I am 100% okay with the limit being 100MB (or even lower) but a serious problem is for V2 the HTTP header Expect: 100-continue is ignored and assumes the client has sent large body.

It is very useful for large body requests to have a serverless function catch that this is an Expect call and respond with an HTTP 307 redirect pointing to a signed PUT URL that is an storage account where the destination is more than happy to receive a multi-GB/TB upload.

That said, of course for Azure this is impossible as they refuse to make their storage account usable and require the impossible task of setting up a regular HTTP client with the knowledge it is uploading to a storage account and set a header; not possible when you are not in control of the client connecting.

Azure is literally the ghetto...