Integrating Hearbeat helmchart to Opensearch?

[sezerozann]

I'm facing with the issue while integrating heartbeat v7.6.2 to opensearch v2.4.0. Also I'm thinking about what did I wrong and trying to check everything in the system. Right now every heartbeat pod is running but pod logs are giving below error

2023-01-18T13:15:05.401Z        ERROR   pipeline/output.go:100  Failed to connect to backoff(elasticsearch(http://i3-logging-opensearch-master:9200)): Connection marked as failed because the onConnect callback failed: Heartbeat requires the default distribution of Elasticsearch. Please update to the default distribution of Elasticsearch for full access to all free features, or switch to the OSS distribution of Heartbeat. 
2023-01-18T13:15:05.401Z        INFO    pipeline/output.go:93   Attempting to reconnect to backoff(elasticsearch(http://i3-logging-opensearch-master:9200)) with 1830 reconnect attempt(s) 
2023-01-18T13:15:05.401Z        INFO    [publisher]     pipeline/retry.go:196   retryer: send unwait-signal to consumer 
2023-01-18T13:15:05.401Z        INFO    [publisher]     pipeline/retry.go:198     done 
2023-01-18T13:15:05.401Z        INFO    [publisher]     pipeline/retry.go:173   retryer: send wait signal to consumer 
2023-01-18T13:15:05.401Z        INFO    [publisher]     pipeline/retry.go:175     done 
2023-01-18T13:15:05.402Z        INFO    elasticsearch/client.go:757     Attempting to connect to Elasticsearch version 7.10.2 
2023-01-18T13:15:18.980Z        INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":13020,"time":{"ms":8}},"total":{"ticks":39380,"time":{"ms":29},"value":39380},"user":{"ticks":26360,"time":{"ms":21}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"8dd7215b-0422-42ac-b1aa-bf476273922a","uptime":{"ms":81870017}},"memstats":{"gc_next":28707184,"memory_alloc":14497424,"memory_total":1186677840},"runtime":{"goroutines":17}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"bytes":731},"write":{"bytes":300}},"pipeline":{"clients":1,"events":{"active":4117,"retry":3}}},"system":{"load":{"1":0.75,"15":0.72,"5":0.79,"norm":{"1":0.1875,"15":0.18,"5":0.1975}}}}}}
2023-01-18T13:15:44.000Z        ERROR   pipeline/output.go:100  Failed to connect to backoff(elasticsearch(http://i3-logging-opensearch-master:9200)): Connection marked as failed because the onConnect callback failed: Heartbeat requires the default distribution of Elasticsearch. Please update to the default distribution of Elasticsearch for full access to all free features, or switch to the OSS distribution of Heartbeat. 
2023-01-18T13:15:44.000Z        INFO    pipeline/output.go:93   Attempting to reconnect to backoff(elasticsearch(http://i3-logging-opensearch-master:9200)) with 1831 reconnect attempt(s) 
2023-01-18T13:15:44.000Z        INFO    [publisher]     pipeline/retry.go:196   retryer: send unwait-signal to consumer 
2023-01-18T13:15:44.000Z        INFO    [publisher]     pipeline/retry.go:198     done 
2023-01-18T13:15:44.000Z        INFO    [publisher]     pipeline/retry.go:173   retryer: send wait signal to consumer 
2023-01-18T13:15:44.000Z        INFO    [publisher]     pipeline/retry.go:175     done 
2023-01-18T13:15:44.001Z        INFO    elasticsearch/client.go:757     Attempting to connect to Elasticsearch version 7.10.2 
2023-01-18T13:15:48.979Z        INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":13020,"time":{"ms":6}},"total":{"ticks":39380,"time":{"ms":9},"value":39380},"user":{"ticks":26360,"time":{"ms":3}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"8dd7215b-0422-42ac-b1aa-bf476273922a","uptime":{"ms":81900017}},"memstats":{"gc_next":28707184,"memory_alloc":14946280,"memory_total":1187126696},"runtime":{"goroutines":17}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"bytes":731},"write":{"bytes":300}},"pipeline":{"clients":1,"events":{"active":4117,"retry":1}}},"system":{"load":{"1":0.71,"15":0.72,"5":0.78,"norm":{"1":0.1775,"15":0.18,"5":0.195}}}}}}

Does anyone know how to connect that heartbeat to opensearch?

[peterzhuamazon]

Move to helm-charts.

[gaiksaya]

[Triage] Hi @sezerozann ,

Is it same as https://github.com/opensearch-project/opensearch-build/issues/3010?