CI: Lower garbage collection load in go tests

[ValarDragon]

trafficstars

Background

On some preliminary estimates, it looks we are spending the majority of real time in our golang tests in CI, on garbage collection.

This was done by comparing local vs CI, and that the summation of the test time numbers in CI logs.

e.g. upgrades v9 taking 4 seconds of real time:

I didn't find any pattern in these, other than this overhead was reliably there for many of the larger time tests, hence I suspect its a GC situation.

Suggested Design

• Find way to measure if this time is from GC
• Can we do something in args to reduce GC overhead in CI
• Reduce Ram pressure from our tests

Acceptance Criteria

• [ ] go CI time becomes more comparable to summation of individual test times. (currently 4 min 20 seconds vs 1 min 30 seconds it should be)

[pysel]

hello @ValarDragon I will look into that if that's ok?

[ValarDragon]

Please do