Not deleting airflow logs for master node

[chetkhatri]

This util dag is not deleting airflow logs for the master node but doing a delete for all worker nodes, please guide to do delete for the master node also where It is Airflow Cluster with Celery Executor. Thanks

[hasher1705]

You can manually set up a cron job to delete log files in the master node. We have got a similar setup as yours. find /home/ubuntu/airflow/logs/*/* -type f -mtime +7 -exec rm -rf {} \; We run this once every day.

[prakshalj0512]

@chetkhatri the DAG task deletes logs on the node on which it is running so that's the reason why only logs on worker nodes are being removed. You can try the new airflow-log-cleanup-pwdless-ssh.py DAG and specify the hosts to run the process on Line 41. Follow the readme for this DAG and make sure you're able to passwordless SSH onto the nodes from the worker nodes.