Memory leak when using the `/metrics` endpoint (PrometheusExporter)

[krynju]

I've identified a memory leak in a long running service coming from OpenTelemetry.

I'm attaching an MWE split off from the main service, which can reproduce the issue identically The example attached is pretty extreme and can get you to ~2GB memory usage in about 10 minutes.

The leak is dependent on measure activity and capturing measures through the /metrics endpoint.

1. metricsspam emulates a running service, which generates some metric activity
2. for emulating metrics capture we use ab to call the /metrics endpoint extensively

Version info: julia 1.9.3, linux OpenTelemetry versions as the attached Manifest.toml or simpler, this repo at commit https://github.com/oolong-dev/OpenTelemetry.jl/commit/4975ecddf521f4fa930d4bc8964c0b8a91301039

subdir="src/api",                      # 0.3.0
subdir="src/sdk",                      # 0.2.1
subdir="src/proto",                    # 0.13.1
subdir="src/exporter/otlp/proto/grpc", # 0.2.1
subdir="src/exporter/prometheus",      # 0.1.4

Reproducer: memleak.zip

Tar contents:

1. run.jl - reproducer code
2. make.jl - code to generate project/manifest with exact package revisions used
3. Project.toml/Manifest.toml

Steps:

1. extract the zip
2. run the reproducer using julia --project=. -e "include(\"run.jl\");init();"
3. run the metrics capture emulation using ab -c 500 -n 1000000 http://localhost:9967/
4. observe endlessly rising memory usage in the process monitoring tool of your choice

Example runtime (~4gb in 25min)