feat: use map instead of loop in array to improve the performance

[weixiao-huang]

Description

This bug fixes #6899

Context

Gloo loop a large size pods array in a large size upstreams when using gloo in a large k8s cluster with much pods and upstreams. This PR use map instead of array to accelerate it.

Checklist:

• [ ] I included a concise, user-facing changelog (for details, see https://github.com/solo-io/go-utils/tree/master/changelogutils) which references the issue that is resolved.
• [x] If I updated APIs (our protos) or helm values, I ran make -B install-go-tools generated-code to ensure there will be no code diff
• [ ] I followed guidelines laid out in the Gloo Edge contribution guide
• [ ] I opened a draft PR or added the work in progress label if my PR is not ready for review
• [x] I have performed a self-review of my own code
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have added tests that prove my fix is effective or that my feature works

[solo-build-bot[bot]]

Waiting for approval from someone in the solo-io org to start testing.

[elcasteel]

Thanks for submitting this! We're going to find a reviewer from Solo to take a look at this although we are currently fairly busy so it may take a bit of time to prioritize. In the mean time, it would be very helpful if you did a couple of things:

• Add a changelog
• Add some supporting documentation for why you think this will improve performance
• Ensure that unit tests pass and if it's appropriate add a new one.

[nfuden]

Sorry for how long this has been. I like the idea but we just need to verify a few things about the behavior change.

For example translation idempotency. Not too hard to check but it just takes some time.

Thanks for your patience!

[weixiao-huang]

Thanks for submitting this! We're going to find a reviewer from Solo to take a look at this although we are currently fairly busy so it may take a bit of time to prioritize. In the mean time, it would be very helpful if you did a couple of things:

• Add a changelog
• Add some supporting documentation for why you think this will improve performance
• Ensure that unit tests pass and if it's appropriate add a new one.

Sorry for my delay. I have added the changelog and ensured the tests pass. However, where should I add the documentation? This changes may only contains code improvement but do not change the user interface. I use map instead of array for accelerating the speed of translation and eds. I have tested it in a large k8s cluster containing about 3w+ pods and 8k+ endpoints (or upstreams). This changes may reduce eds time from 30s to 0.5s and reduce time of translateClusterSubsystemComponents from 30s to 3s.

[weixiao-huang]

Sorry for how long this has been. I like the idea but we just need to verify a few things about the behavior change.

For example translation idempotency. Not too hard to check but it just takes some time.

Thanks for your patience!

I think the behavior is valid that I only change the for from loop to search in a map

[solo-changelog-bot[bot]]

Issues linked to changelog: https://github.com/solo-io/gloo/issues/6899

[weixiao-huang]

Is there any update of this PR?

[sam-heilbron]

/test

[elcasteel]

/test