Memory footprint featureflag service

[jsirianni]

Question

Is it possible to reduce featureflagservice's memory footprint? Everything in the stack is pretty light except for featureflagservice, which consumes 1.5GB+ memory.

CONTAINER ID   NAME                                                  CPU %     MEM USAGE / LIMIT   MEM %     NET I/O           BLOCK I/O         PIDS
dfd7316ca2de   opentelemetry-demo-webstore_featureflagservice_1      0.12%     1.664GiB / 2GiB     83.18%    65.8kB / 32kB     0B / 0B           85
b86084052e3e   opentelemetry-demo-webstore_prometheus_1              0.00%     37.17MiB / 256MiB   14.52%    25.9kB / 9.84kB   0B / 41kB         32
10f9eb393ba9   opentelemetry-demo-webstore_grafana_1                 0.00%     43.16MiB / 256MiB   16.86%    15kB / 3.55kB     0B / 23.1MB       34
02b1eb619126   opentelemetry-demo-webstore_ffs_postgres_1            0.00%     43.85MiB / 64MiB    68.51%    35.8kB / 63.2kB   36.9kB / 68.1MB   17
39d8ae11bfcb   opentelemetry-demo-webstore_redis-cart_1              0.06%     3.105MiB / 64MiB    4.85%     43.4kB / 22.7kB   0B / 0B           5
c49dc0a5ce1d   opentelemetry-demo-webstore_jaeger_1                  0.07%     26.73MiB / 256MiB   10.44%    2.18MB / 31.5kB   0B / 0B           6
0908487a3491   opentelemetry-demo-webstore_otelcol_1                 0.18%     37.02MiB / 64MiB    57.85%    1.71MB / 2.26MB   0B / 0B           31
8d57f6cbb3bd   opentelemetry-demo-webstore_productcatalogservice_1   0.27%     8.148MiB / 64MiB    12.73%    330kB / 829kB     0B / 0B           20
db1e270a8723   opentelemetry-demo-webstore_emailservice_1            0.08%     44.72MiB / 128MiB   34.94%    11kB / 9.52kB     0B / 4.1kB        7
659fcab951c0   opentelemetry-demo-webstore_shippingservice_1         0.04%     6.773MiB / 64MiB    10.58%    39.3kB / 48.9kB   0B / 0B           14
f4a4d6e4d7c9   opentelemetry-demo-webstore_cartservice_1             0.27%     28.54MiB / 64MiB    44.60%    88.7kB / 203kB    0B / 0B           31
5ccb461d1005   opentelemetry-demo-webstore_adservice_1               0.91%     146.9MiB / 512MiB   28.69%    37kB / 74kB       0B / 385kB        26
2b5447c72823   opentelemetry-demo-webstore_paymentservice_1          0.00%     24.5MiB / 64MiB     38.28%    12.3kB / 12.8kB   0B / 0B           11
704d1b39d491   opentelemetry-demo-webstore_currencyservice_1         1.04%     27.39MiB / 64MiB    42.79%    160kB / 196kB     0B / 0B           11
fbf83d199fd7   opentelemetry-demo-webstore_checkoutservice_1         0.00%     8.062MiB / 64MiB    12.60%    66.5kB / 114kB    0B / 0B           19
2beffbeff6cf   opentelemetry-demo-webstore_recommendationservice_1   0.77%     23.61MiB / 64MiB    36.88%    167kB / 121kB     0B / 45.1kB       9
f1b551281704   opentelemetry-demo-webstore_frontend_1                0.90%     12.37MiB / 64MiB    19.32%    560kB / 2.37MB    0B / 0B           27
863c93a87c80   opentelemetry-demo-webstore_loadgenerator_1           0.76%     40.72MiB / 64MiB    63.63%    911kB / 91.2kB    0B / 188kB        7

Context: My fork sets deploy.resources.limits for each service, and the featureflag service refuses to run unless given at least 1GB memory.

[cartersocha]

@tsloughter @maciej-szlosarczyk fyi

[maciej-szlosarczyk]

I tried it locally and it did run with 256MB of memory, but I saw an issue with the order of dependencies (postgres did not start when the service tried to use it). I will take a closer look some time later this week.

@jsirianni if you could send the stdout logs from an unsuccessful run that would be of great help 🙇

[tsloughter]

Sorry, only seeing this now. It should not use that much ram in the slightest. I see no reason it shouldn't work with 256MB of ram, so will have to investigate why it would be using so much.

[tsloughter]

And yea, as @maciej-szlosarczyk asked, the logs would be helpful.

[cartersocha]

@jsirianni could you provide the logs? We haven’t been able to replicate any memory issues on other hardware & are a bit stumped

[cartersocha]

Closing for now. We can re-open if this is replicated on a current version