feat: allow list items to be processed in parallel

[shady-canva]

This PR introduces a new setting which allows items returned by ListPager to be processed in parallel.

Currently, items are processed sequentially per resource. This is a problem if populateResourceInfoHandler is expensive. We internally have a custom resource which takes 1.5 milliseconds in the populateResourceInfoHandler function due to a custom lua healthcheck script and a few ignored labels which are removed during diff normalisation. With over 50,000 of these custom resources, that's a 75 second overhead.

With a parallelism of 8, we've seen a page of 500 items go from ~750ms to ~100ms. We've also seen the processing of a full list complete at roughly the same time as ListPager completes. Previously, processing of pages constantly lagged behind the retrieval.

gobench results:

BenchmarkSync/weight=1,overhead=100μs-12          	      68	  16072692 ns/op
BenchmarkSync/weight=2,overhead=100μs-12          	     139	   8585231 ns/op
BenchmarkSync/weight=4,overhead=100μs-12          	     242	   4905336 ns/op
BenchmarkSync/weight=8,overhead=100μs-12          	     302	   3939709 ns/op

BenchmarkSync/weight=1,overhead=500μs-12          	      19	  63097167 ns/op
BenchmarkSync/weight=2,overhead=500μs-12          	      38	  31135898 ns/op
BenchmarkSync/weight=4,overhead=500μs-12          	      75	  15546984 ns/op
BenchmarkSync/weight=8,overhead=500μs-12          	     141	   8420676 ns/op

Before and after:

[sonarqubecloud[bot]]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 83.87097% with 10 lines in your changes missing coverage. Please review.

Project coverage is 47.67%. Comparing base (8849c3f) to head (4e4d254). Report is 52 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/cache/cluster.go	82.75%	7 Missing and 3 partials :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #738      +/-   ##
==========================================
- Coverage   54.26%   47.67%   -6.59%     
==========================================
  Files          64       64              
  Lines        6164     6611     +447     
==========================================
- Hits         3345     3152     -193     
- Misses       2549     3199     +650     
+ Partials      270      260      -10     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.