Flower

[edbizarro]

[domdip]

FYI - in latest run, Flower didn't start. Probably need to pass -D to webserver - daemonize it so 2nd part of command runs

[edbizarro]

Thanks! yeah, you're right. I'll change the script

[edbizarro]

Found out that flower does not support SQS

So I'm moving to redis instead

[domdip]

FYI what actually needs to happen is something like this (used in my case where I'm putting the scheduler and webserver on one host).

#!/usr/bin/env bash
if [ "\$AIRFLOW_ROLE" == "SCHEDULER" ]
then exec airflow scheduler -n 10
elif [ "\$AIRFLOW_ROLE" == "WEBSERVER" ]; then 
    exec airflow webserver -D 
    exec airflow scheduler -D -n 10
elif [ "\$AIRFLOW_ROLE" == "WORKER" ]
then exec airflow worker
else echo "AIRFLOW_ROLE value unknown" && exit 1
fi

exec spawns things in a new (dedicated) process so it can't have bash helpers like &&, will need to be called twice.

[edbizarro]

Yes, i did that, but flower throws errors wich led me to that issue saying flower doesnt support SQS

[domdip]

Actually, be careful using that approach. This may not be an issue with Flower but Scheduler quits frequently and is managed by systemctl. Putting two things under that same script will mean that systemctl will wait for all of them to exit before restarting either.

I had to rewrite this a lot for my use case (scheduler + webserver on same host) ultimate setting up a systemctl for each of them, with slightly different service environments.