[TT-5186] Add a global request body size limit

[PatrickTaibel]

Description

Adds a global request body size limit to the config. For incoming requests the Content-Length will be checked against the limit. If it's set, the limit enabled and the Content-Length exceeds the limit a status 413 (Request Entity Too Large) will be returned and the further handling aborted. Additionally, a http.MaxBytesReader is applied to the request body to force any reads to be limited to the global server limit (e.g. when the Content-Length is not set or manipulated).

Related Issue

https://github.com/TykTechnologies/tyk/issues/3976

Motivation and Context

Tyk API Gateway copies the whole request into memory at the beginning of the request handling. Large requests could fill up memory if they are not blocked. At the moment this would need to be done by an additional load balancer or proxy in front.

How This Has Been Tested

As long as the limit is not enabled (which is the default case) there is no change in behavior. Therefore, no existing installations are affected by the change. In case it's enabled only requests larger then the configured limit are affected.

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

• [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
• [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
• [ ] My change requires a change to the documentation.
  • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
  • [ ] If new config option added, ensure that it can be set via ENV variable
• [ ] I have updated the documentation accordingly.
• [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
• [ ] When updating library version must provide reason/explanation for this update.
• [ ] I have added tests to cover my changes.
• [x] All new and existing tests passed.
• [x] Check your code additions will not fail linting checks:
  • [x] go fmt -s
  • [x] go vet

[buger]

Nice!

Clean and simple!

Would also want add simple test to cover it?

The test can look like this (have not tested):

func TestRequestBodyLimit(t *testing.T) {
	ts := StartTest(nil)
	defer ts.Close()

	globalConf := ts.Gw.GetConfig()
	globalConf. MaxRequestBodySize = 1
	ts.Gw.SetConfig(globalConf)

	ts.Gw.BuildAndLoadAPI(func(spec *APISpec) {
		spec.UseKeylessAccess = true
	})

	_, _ = ts.Run(t, test.TestCase{
		Path: "/sample/", Method: "POST", Data: "test", Code: http.StatusRequestEntityTooLarge,
	})
}

[PatrickTaibel]

I've added a small test for this. Thank you very much for the pretty much finished test snippet! I've changed the way to set the config though. With your way it seems to not restart/reinitialize the http handler and won't reread the HttpServerOptions. Therefore, I've set the config function in the StartTest call for this.

[titpetric]

@PatrickTaibel I'm noticing this is used as the base of the following PR: https://github.com/TykTechnologies/tyk/pull/3979 - can you please include those changes here, and close that PR?

[PatrickTaibel]

@titpetric Those two PRs are independent of each other (and both branches are directly based on master). This one adds a global serverwide hard limit on the request body size. The other PR fixes enabling the request size limit middleware which is API specific. In my opinion, those are two different things. But if it's easier to handle on your side, I can for sure combine both changes here.

[titpetric]

My mistake, it seemed like a patch of this one. Will do housekeeping there too, thanks :)