airflow
airflow copied to clipboard
Fetch intermediate log async GKEStartPod
This PR introduces a parameter that enables the retrieval of intermediate logs for the GKEStartPod asynchronous operator.
- Add param
last_log_time
andlogging_interval
in GKEStartPodTrigger serialize - Add optional param
last_log_time
in methodinvoke_defer_method
Example DAG:
start_pod = GKEStartPodOperator(
task_id="start_pod",
project_id=PROJECT_ID,
location=LOCATION,
cluster_name=GKE_CLUSTER_NAME,
do_xcom_push=True,
namespace=GKE_NAMESPACE,
image="ubuntu:jammy",
cmds=["sh", "-c", "timeout 300 bash -c 'while true; do echo \"meow\"; sleep 30; done'"],
name="test-sleep",
in_cluster=False,
on_finish_action="delete_pod",
deferrable=True,
get_logs=True,
logging_interval=5,
gcp_conn_id=GCP_CONN_ID
)
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst
or {issue_number}.significant.rst
, in newsfragments.
Any updates on this?
Any updates on this?
I'll try address the pending review by end of this week
Bump on this 🙏
@pankajastro Any ETA on this? Thanks
@pankajastro Any ETA on this? Thanks
CI is green now. Hopefully, we will merge soon. In the meantime, could you please test it once?