Align integration & apm-server binary config for read timeout

[axw]

The integration defaults the read timeout to 3600s (which is ridiculously high?):

https://github.com/elastic/apm-server/blob/5f923f43e62bb172540cfe54f6a7fc0bc2d134d1/apmpackage/apm/manifest.yml#L108-L110

Meanwhile the apm-server binary default is 30s:

https://github.com/elastic/apm-server/blob/5f923f43e62bb172540cfe54f6a7fc0bc2d134d1/internal/beater/config/config.go#L138

We should review the default, and align these.

[simitt]

Wouldn't this be a breaking change in either of them?

[axw]

In theory maybe, but is it realistic for a request to be open for 3600s?

[axw]

There's some documentation about the timeouts at https://www.elastic.co/guide/en/apm/guide/current/common-problems.html#io-timeout. Regardless of whether we change either one, we should link from the config to that docs section, and explain why they need to be increasing -- not just that need to be increasing.

[kruskall]

I recall the hight timeout had something to do with the proxy on elastic cloud (I might be wrong).

@axw could you clarify what should we use as the timeout for both (30s vs 3600s) and if we need to reach out to other parties that might be affected by this ?

[axw]

could you clarify what should we use as the timeout for both (30s vs 3600s) and if we need to reach out to other parties that might be affected by this ?

@kruskall I don't know. It's not intuitive to me why the timeouts should be increasing, vs. aligned. If the client would timeout after 10s, then I don't see how it would matter if the load balancer could handle a request for anything greater than 10s.

If they need to be increasing, then out of those two we should go with 3600s. But we need to understand and explain why, and document that.

If they don't need to be increasing, we may want to change the defaults completely.