Load Provider Plugins in Development

[jscheffl]

As I am starting to build a PoC and experience code in and for AIP-69 I realized that provider packages from the mono-repo do not load plugins when starting the system via breeze. Plugins are correctly published via entry-point definition if packages are released but are missing in the global pyproject.toml. Mainly also because they should not be contained there.

Following a Slack discussion in https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1715545743922549 with @potiuk I attempted and succeeded in extending the hatch_build.py. With this PR the plugins defined in the provider packages are dynamically extended for the development setup, means that when starting Airflow with breeze, the included "hive" and "openlineage" plugin are loaded by default. (Any maybe if Romte Executor will bring nother also this...

How to test?

• Use the branch and start Airflow with breeze start-airflow --answer y
• In the tmux started run airflow plugins and see hive and openlineage are added from source automatically.

Note: Needed to extend the "provider_dependencies.json" to have the plugin information contained there as well as hatch loads the other details from there as well. First wanted to start another second JSON... but actually the provider dependencies is not cleanly a dependency, so added it there.