dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Improvement] Subworkflow task use t_ds_task_instance_context store runtime context

Open ruanwenjun opened this issue 11 months ago • 3 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Description

All tasks should use t_ds_task_instance_context to store its runtime context, runtime context represent its runtime info , once its runtime context changed, it should tell master to update its runtime context.

When task failover should copy runtime context to new task instance which is generated by failover and provide it's runtime context to task plugin to do failover, so task plugin should provide interface how to deserialze/serialize runtime context, and how to use runtime context to do failover.

image

Are you willing to submit a PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

ruanwenjun avatar Jan 01 '25 03:01 ruanwenjun

The TaskInstanceContext is serialized as JSON string, this class contains the context used or generated by task instance, is an interface, each task can have its own type.

On Fri, Jan 3, 2025 at 5:13 PM lile @.***> wrote:

May I ask, will you also write the calculated date parameters into t_ds_task_instance_context? And would you do it in the form of fields or JSON?

Lately, I've been thinking about how to optimize the logic where DEPENDENT task uses scheduleTime. The tricky parts right now are database efficiency issues and business continuity problems caused by new objects and logic, especially the latter, which will make upgrades quite challenging for 24/7 operation scenarios.

So, could you please give a hint about the next design step?

— Reply to this email directly, view it on GitHub https://github.com/apache/dolphinscheduler/issues/16938#issuecomment-2568911077, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFLAR2VHDLWJCJ3F6SN6CJ32IZID5AVCNFSM6AAAAABUOFWNX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRYHEYTCMBXG4 . You are receiving this because you were assigned.Message ID: @.***>

ruanwenjun avatar Jan 09 '25 07:01 ruanwenjun

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Feb 11 '25 00:02 github-actions[bot]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Mar 14 '25 00:03 github-actions[bot]