aws-airflow-stack
aws-airflow-stack copied to clipboard
villasv
Turbine: the bare metals that gets you Airflow
Has anyone come across this issue where the worker crashes/terminates with this error Traceback (most recent call last): File "/home/airflow/.local/lib/python3.6/site-packages/celery/worker/worker.py", line 208, in start self.blueprint.start(self) File "/home/airflow/.local/lib/python3.6/site-packages/celery/bootsteps.py", line 119, in...
I deployed this cloudformation without changes, and successfully got to the dashboard after adding a security group change. I was able to write and deploy my own DAG, and schedule...
and what other steps do i need to do to start submitting jobs? this isn't clear to me. thanks!
Merged in 1.10.12: https://github.com/apache/airflow/pull/9645
Hello, the scheduler has failed and then it keeps trying to get recreated without success: ``` [ 22.583701] cloud-init[2662]: Cloud-init v. 19.3-2.amzn2 running 'modules:final' at Tue, 23 Jun 2020 20:16:58...
Hi vilasv, First of - huge fan of your work. I don't have many running dags and need only between 0 - 1 worker running at any time but when...
For the moment, the setup script does an `aws s3 sync`, but that will show up on my billing soon enough.
Following the best practices described in the [aws blog](https://aws.amazon.com/blogs/mt/using-state-manager-over-cfn-init-in-cloudformation-and-its-benefits/)
A common requirement. Also relevant to show that #172 is not a reason to fork the project.