Fix deferrable mode for DataflowTemplatedJobStartOperator and DataflowStartFlexTemplateOperator

[e-galan]

This fixes the deferrable mode for DataflowTemplatedJobStartOperator and DataflowStartFlexTemplateOperator. Previously the deferrable mode was implemented in a way that made most of the task execution run in the sync mode, and only after the execution was finished, the control was passed to the trigger, where it only checked the job status (that was by this time completed) and then immediately returned.

List of changes:

• Add new methods to DataflowHook to be used for the deferrable mode. The methods will start a Dataflow job and then exit returning the job data. The existing hook methods are left without changes and are still used for the sync mode.
• Add code to update XCOM with Dataflow job_id in both sync and deferrable modes.
• Update unit tests
• Small refactoring of data types and function parameters

---

^ Add meaningful description above Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[e-galan]

Hi @eladkal @Lee-W @potiuk ! Could you take a look at the PR?

[potiuk]

Could you please rebase and resolve conflict as well @e-galan ? 0

[e-galan]

Could you please rebase and resolve conflict as well @e-galan ? 0

Sure @potiuk , PR is now rebased and the conflict resolved.

[potiuk]

Don't know all the details but it seems good for review.