Too high memory usage on history service

[koolay]

Expected Behavior

Require less memory usage.

Actual Behavior

container resources:

server metrics:

go tool pprof "http://localhost:7936/debug/pprof/heap"

Steps to Reproduce the Problem

1. create one thousand workflows

temporal-dynamic-config

apiVersion: v1
data:
 dynamic_config.yaml: |-
   history.cacheInitialSize:
   - value: "10"
   history.cacheMaxSize:
   - value: "50"
   history.cacheTTL:
   - value: "0.5"
   history.eventsCacheInitialSize:
   - value: "10"
   history.eventsCacheMaxSize:
   - value: "50"
   matching.useOldRouting:
   - value: "false"
kind: ConfigMap
metadata:
 annotations:
   meta.helm.sh/release-name: temporal
   meta.helm.sh/release-namespace: temporal
 creationTimestamp: "2022-11-09T04:02:55Z"
 labels:
   app.kubernetes.io/instance: temporal
   app.kubernetes.io/managed-by: Helm
   app.kubernetes.io/name: temporal
   app.kubernetes.io/part-of: temporal
   app.kubernetes.io/version: 1.20.2
   helm.sh/chart: temporal-0.20.2
 name: temporal-dynamic-config
 namespace: temporal
 resourceVersion: "64727569"
 uid: 01925a28-4772-42e9-891e-bdb467bf496f

Specifications

• Version: temporalio/server:1.20.2
• Platform: k8s

[leididi]

encountered the same problem；

Name:         temporal-dynamic-config
Namespace:    temporal
Labels:       app.kubernetes.io/instance=temporal
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=temporal
              app.kubernetes.io/part-of=temporal
              app.kubernetes.io/version=1.20.2
              helm.sh/chart=temporal-0.20.2
Annotations:  meta.helm.sh/release-name: temporal
              meta.helm.sh/release-namespace: temporal

Data
====
dynamic_config.yaml:
----
history.cacheInitialSize:
- value: "10"
history.cacheMaxSize:
- value: "50"
history.cacheTTL:
- value: "0.1"
history.eventsCacheInitialSize:
- value: "10"
history.eventsCacheMaxSize:
- value: "50"
matching.useOldRouting:
- value: "false"

[trangtran-ct]

Hi, we experienced the same problem. Any update on this? Thanks a lot.

[DhirajBhakta]

Noticing the same issue. Could you help us understand if the history service detects available memory(available to the pod) and automatically set max cache size to available value?