Replace JSON library for targeted parsing of the Kubelet response
The Kubelet response can sometimes be quite large (i.e. larger than 1MiB). This has traditionally caused some performance problems.
#4231 replaced the use of the standard library encoding/json package with github.com/valyala/json which improved the CPU utilization by a fair margin. However, valyala/json has some fairly high memory requirements, as well as some bugs that make it retain memory longer than necessary. This has led to OOM conditions for SPIRE Agent (#5067). Unfortunately, valyala/json also seems to no longer be actively maintained, making reliance on it a risk to the project.
We should replace the valyala/json library with one that:
- Allows for targeted parsing of relevant pod information from the kubelet response (workload attestor is only interested in a single pod/container)
- Has favorable CPU/memory utilization
- Is actively maintained
I'd suggest trying to go with https://github.com/go-json-experiment/json, have a look also at its goals.