finch icon indicating copy to clipboard operation
finch copied to clipboard

Enable running local AWS Managed Workflows for Apache Airflow

Open austinvazquez opened this issue 1 year ago • 1 comments

What is the problem you're trying to solve?. I am unable to run https://github.com/aws/aws-mwaa-local-runner when swapping docker out for Finch.

Describe the feature you'd like As finch user, I would like to be able to run AWS Managed Workflow Apache Airflow (MWAA) Directed Acyclic Graph (DAG) locally using https://github.com/aws/aws-mwaa-local-runner

Additional context While the work to support running aws-mwaa-local-runner using either Docker or Finch should be done in https://github.com/aws/aws-mwaa-local-runner, this issue is to track the feature gaps which would enable aws-mwaa-local-runner to run on Finch.

Tasks

  • [x] #571
  • [ ] #572

austinvazquez avatar Sep 14 '23 22:09 austinvazquez

Checking how apache airflow uses container runtime. Finch/nerdctl use local build context so it is not really relevant to compress the build context. We can probably add it as do nothing in nerdctl but checking if it can be done in downstream.

Shubhranshu153 avatar May 13 '24 16:05 Shubhranshu153