updating function config: RequestEntityTooLargeException

[timsuchanek]

• [x] I am running the latest version. (up upgrade)
• [x] I searched to see if the issue already exists.
• [x] I inspected the verbose debug output with the -v, --verbose flag.
• [x] Are you an Up Pro subscriber? Yes

When deploying an app with up, that we have been deploying successfully for months, we suddenly get this error:

   ⠧ 0s      DEBU loaded env vars duration=730
     0s      DEBU loaded env vars duration=730

   ⠧ 0s      DEBU updating function
   ⠧ 44ms    DEBU updating function
     78ms    DEBU updating function

   ⠇ 0s      DEBU event platform.function.update map[commit:e092dfe stage:production region:eu-west-1]
     0s      DEBU event platform.function.update map[commit:e092dfe stage:production region:eu-west-1]

   ⠇ 0s      DEBU event platform.deploy.complete map[commit:e092dfe stage:production region:eu-west-1 duration:4.296283736s version:]
     0s      DEBU event platform.deploy.complete map[commit:e092dfe stage:production region:eu-west-1 duration:4.296283736s version:]

   ⠇ 0s      DEBU event deploy.complete map[commit:e092dfe stage:production duration:19.681194015s]

     Error: deploying: eu-west-1: updating function config: RequestEntityTooLargeException: Request must be smaller than 5120 bytes for the UpdateFunctionConfiguration operation
	status code: 413, request id: 00d16139-d85d-11e8-b5fe-cbd24e1c9fef

Exited with code 1

The odd thing is, that we get this even when we remove env vars (I was suspecting them to be the cause for this) and also if we remove old deployments - both doesn't help. The size of the deployment also didn't change. In the AWS I also can't see anything that could increase this UpdateFunctionConfiguration.

@tj is there anything that comes to your mind that could cause this?

Let me know, if you need more information.

Keep up the good work, thanks!

[tj]

Oh interesting, yeah I would definitely suspect env vars, nothing else in the function config is of a variable size really. In any case 5k seems really small to me for env vars, so I'm going to forward that to the AWS team, but aside from that I'm not really sure.

This is the entire config, so unless you have like 100 subnets it should be ok haha.

	_, err = c.UpdateFunctionConfiguration(&lambda.UpdateFunctionConfigurationInput{
		FunctionName: &p.config.Name,
		Handler:      &p.handler,
		Runtime:      &p.runtime,
		Role:         &p.config.Lambda.Role,
		MemorySize:   aws.Int64(int64(p.config.Lambda.Memory)),
		Timeout:      aws.Int64(int64(p.config.Proxy.Timeout + 3)),
		Environment:  env,
		VpcConfig:    p.vpc(),
	})

[timsuchanek]

Thanks! I checked the VPC config, it's empty. But yeah I also checked the env vars, we definitely have more than 5k chars in there, but they're all required to run the application properly and I don't want to give up on security.

It's really weird that this comes out of nowhere, so there must be a problem from the AWS side. I even tried to redeploy an old deploy that used to work, getting the same error now. I'll also contact the support to accelerate it as we can't deploy an important service right now and are blocked in development there.

[tj]

I think you should be ok deploying after up upgrade -t 0.7.3 to grab an older version of Up.

In a more recent version I changed it from loading at runtime from SSM (costs more and has a ~600ms latency hit), to the "built-in" Lambda env vars, but I didn't realize the limit was so low.

If this becomes a common thing I'll probably have to revert that, 4k still seems incredibly low to me, they shouldn't support env vars if they're not going to have space for them.

[timsuchanek]

Thanks! Yes with up upgrade -t 0.7.3 we're good to go :) Now we can finally deploy again 🙏

[tj]

Damn, sorry about that! I'll keep bugging them about raising that limit