Startup takes around 30s

[pavolloffay]

Startup in api-anomaly-detector service in Traceable takes around 30s at least based on the agent logs

Picked up JAVA_TOOL_OPTIONS: -XX:InitialRAMPercentage=50.0 -XX:MaxRAMPercentage=75.0 -javaagent:/mnt/hypertrace/hypertrace-agent-all.jar
[main] INFO org.hypertrace.agent.core.config.HypertraceConfig - Config loaded: {"serviceName":"api-anomaly-detector","reporting":{"endpoint":"http://collector:9411/api/v2/spans","opa":{"endpoint":"http://aaa:8181/","pollPeriodSeconds":30}},"dataCapture":{"httpHeaders":{"request":true,"response":true},"httpBody":{"request":true,"response":true},"rpcMetadata":{"request":true,"response":true},"rpcBody":{"request":true,"response":true},"bodyMaxSizeBytes":131072},"propagationFormats":["TRACECONTEXT"]}
[opentelemetry.auto.trace 2021-01-09 06:41:22:455 +0000] [main] INFO org.hypertrace.agent.core.config.HypertraceConfig - Config loaded: {"serviceName":"api-anomaly-detector","reporting":{"endpoint":"http://collector.aaa:9411/api/v2/spans","opa":{"endpoint":"http://aaa:8181/","pollPeriodSeconds":30}},"dataCapture":{"httpHeaders":{"request":true,"response":true},"httpBody":{"request":true,"response":true},"rpcMetadata":{"request":true,"response":true},"rpcBody":{"request":true,"response":true},"bodyMaxSizeBytes":131072},"propagationFormats":["TRACECONTEXT"]}
[opentelemetry.auto.trace 2021-01-09 06:41:41:350 +0000] [main] INFO io.opentelemetry.javaagent.tooling.OpenTelemetryInstaller - Installed span exporter: io.opentelemetry.exporter.zipkin.ZipkinSpanExporter
[opentelemetry.auto.trace 2021-01-09 06:41:41:562 +0000] [main] INFO io.opentelemetry.javaagent.tooling.PropagatorsInitializer - Added tracecontext propagator
Hypertrace agent started, version: 0.7.5
2021-01-09 06:41:51.959 [main] INFO  o.h.c.s.PlatformServiceLauncher - Trying to start PlatformService ...

[pavolloffay]

Here are log from the query service 44s:

Picked up JAVA_TOOL_OPTIONS: -XX:InitialRAMPercentage=50.0 -XX:MaxRAMPercentage=75.0 -javaagent:/mnt/hypertrace/hypertrace-agent-all.jar
[main] INFO org.hypertrace.agent.core.config.HypertraceConfig - Config loaded: {"serviceName":"query-service","reporting":{"endpoint":"http://collector.traceableai:9411/api/v2/spans","opa":{"endpoint":"http://opa.traceableai:8181/","pollPeriodSeconds":30}},"dataCapture":{"httpHeaders":{"request":true,"response":true},"httpBody":{"request":true,"response":true},"rpcMetadata":{"request":true,"response":true},"rpcBody":{"request":true,"response":true},"bodyMaxSizeBytes":131072},"propagationFormats":["TRACECONTEXT"]}
[opentelemetry.auto.trace 2021-01-08 17:38:23:811 +0000] [main] INFO org.hypertrace.agent.core.config.HypertraceConfig - Config loaded: {"serviceName":"query-service","reporting":{"endpoint":"http://collector.traceableai:9411/api/v2/spans","opa":{"endpoint":"http://opa.traceableai:8181/","pollPeriodSeconds":30}},"dataCapture":{"httpHeaders":{"request":true,"response":true},"httpBody":{"request":true,"response":true},"rpcMetadata":{"request":true,"response":true},"rpcBody":{"request":true,"response":true},"bodyMaxSizeBytes":131072},"propagationFormats":["TRACECONTEXT"]}
[opentelemetry.auto.trace 2021-01-08 17:39:07:507 +0000] [main] INFO io.opentelemetry.javaagent.tooling.OpenTelemetryInstaller - Installed span exporter: io.opentelemetry.exporter.zipkin.ZipkinSpanExporter
[opentelemetry.auto.trace 2021-01-08 17:39:07:908 +0000] [main] INFO io.opentelemetry.javaagent.tooling.PropagatorsInitializer - Added tracecontext propagator
Hypertrace agent started, version: 0.7.5

[pavolloffay]

First, we should see if there is any regression between the upstream build and HT and then try to figure out how to improve the startup time.

[ravisingal]

one of the possible reason could be cpu throttling. in case of api-anomaly-detector, it has cpu request=0.25 and limit=0.5. How much cpu does it need?

[pavolloffay]

How much cpu does it need?

There is no minimum required CPU, however as you pointed out restricted resources can affect startup time.