spire icon indicating copy to clipboard operation
spire copied to clipboard

Replace JSON library for targeted parsing of the Kubelet response

Open azdagron opened this issue 1 year ago • 1 comments

The Kubelet response can sometimes be quite large (i.e. larger than 1MiB). This has traditionally caused some performance problems.

#4231 replaced the use of the standard library encoding/json package with github.com/valyala/json which improved the CPU utilization by a fair margin. However, valyala/json has some fairly high memory requirements, as well as some bugs that make it retain memory longer than necessary. This has led to OOM conditions for SPIRE Agent (#5067). Unfortunately, valyala/json also seems to no longer be actively maintained, making reliance on it a risk to the project.

We should replace the valyala/json library with one that:

  1. Allows for targeted parsing of relevant pod information from the kubelet response (workload attestor is only interested in a single pod/container)
  2. Has favorable CPU/memory utilization
  3. Is actively maintained

azdagron avatar Apr 30 '24 20:04 azdagron

I'd suggest trying to go with https://github.com/go-json-experiment/json, have a look also at its goals.

adeinega avatar May 02 '24 03:05 adeinega