Rest API - Source File Upload

[necatiarslan]

Description

It is good to have an API to upload source code files to the Airflow server to make deployments.

Use case/motivation

I am the developer of the Airflow VSCode extension, which developers use to build and test Airflow DAGs locally within VSCode. Currently, before running a DAG for testing, the source code must be deployed to the Airflow server. Developers often rely on their own deployment solutions, tailored to their specific development environments. However, if an API were provided to upload source code files directly, the VSCode extension could automatically upload any changed files and run the DAGs, streamlining the testing process. This is one of the most requested features by users of the extension.

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[jscheffl]

Uploading of DAGs was not implemented by intend especially in regards of considerations of security as far as I understand - in the past. I understand for interactive development and debugging this feature might be useful. But as it is a structural change of how Airflow works, I recommend doing this via AIP (Airflow Improvement Process).

There are recently some plans to change DAG deployment in Ariflow 3 under the AIP-66, can you take a look and check if this is the "right" direction? --> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=294816356&src=contextnavpagetreemode

There is a bit more complexity behind the scene. Else it might be a valid topic to be added as plans for future changes towards Ariflow 3. --> https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3+Workstreams