.ops: add ecamp3-logging

[BacLuc]

Finally i have a branch again which works. Now with a lot less yaml.

This adds the EFFK (Elasticsearch, fluentbit, fluentd, Kibana) stack to the ops folder. Kibana and Elasticsearch taken from https://www.digitalocean.com/community/tutorials/how-to-set-up-an-elasticsearch-fluentd-and-kibana-efk-logging-stack-on-kubernetes

For the full story, see: https://github.com/ecamp/ecamp3/issues/4775

[usu]

Nice, thank you @BacLuc

I was not that deeply involved in the discussion in #4775. So this is now a mix between option 1 and 2? Do you have an estimation, how much additional nodes we'd need to run it? Is this deployed to dev already to test out?

[BacLuc]

I was not that deeply involved in the discussion in #4775. So this is now a mix between option 1 and 2?

Yes, but with the fluent operator and fluentbit and fluentd instead of filebeat. You see the list of the tried options here: https://github.com/ecamp/ecamp3/issues/4775#issuecomment-2080457061 This is the last one

Do you have an estimation, how much additional nodes we'd need to run it?

Currently on dev it looks like this:

NAME	CPU(cores)	MEMORY(bytes)
elasticsearch-cluster-0	18m	977Mi
fluent-bit-4mh29	4m	56Mi
fluent-bit-b92zd	3m	52Mi
fluent-bit-fdmzw	3m	47Mi
fluent-bit-s5rgz	4m	48Mi
fluent-operator-7c5d78987c-tbx6l	1m	49Mi
fluentd-0	6m	155Mi
kibana-7cc44b6d66-jllsq	86m	523Mi
Total		1907Mi

So we would need another node. Maybe 2 if elasticsearch contains a lot of logs, we will see.

Is this deployed to dev already to test out?

Yes, you can do a port forward with k port-forward services/kibana 5601:5601