document how to use Elasticsearch for log persistence

[thesuperzapper]

Currently, we describe using PVCs and S3/GCS in "How to persist airflow logs?". However, airflow also supports Elasticsearch, so we should add an "Option 3" which describes this.

The docs for airflow's Elasticsearch provider are here: https://airflow.apache.org/docs/apache-airflow-providers-elasticsearch/stable/logging.html

[ArshiAAkhavan]

thanks i did manage to establish remote logging via elasticsearch+filebeat+logstash in k8s i can help you in that regard

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[thesuperzapper]

Bumping for bot.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[DevFontes]

Greetings everyone.

I'm using chart in version 1.13.1 deploying to Kubernetes RKE2 1.27 with Longhorn as storage class.

When enabling elasticsearch.enabled = true the airflow does not go up without specific logs.

The scheduler and webserver show "................" in the logs and the git-sync container is not available.

Does anyone have any ideas on how to resolve this?