[Core feature] Emit Other CloudEvents

[blaketastic2]

Motivation: Why do you think this is important?

Currently Flyte emits WorkflowExecutionEventRequest, TaskExecutionEventRequest and NodeExecutionEventRequest events that allow external systems to act to state change. We are interested in other state changes in Flyte such as when a Workflow / LaunchPlan is created or updated.

Goal: What should the final outcome look like, ideally?

Looking through the protocol documentation, we've filtered out all requests that cause state change and subsequently are candidates to be emitted as events:

• TaskCreateRequest
• WorkflowCreateRequest
• LaunchPlanCreateRequest
• LaunchPlanUpdateRequest
• ExecutionCreateRequest
• ExecutionRelaunchRequest
• ExecutionRecoverRequest
• ExecutionUpdateRequest
• ExecutionTerminateRequest
• ProjectRegisterRequest
• ProjectDomainAttributesUpdateRequest
• ProjectDomainAttributesDeleteRequest
• ProjectAttributesUpdateRequest
• ProjectAttributesDeleteRequest
• WorkflowAttributesUpdateRequest
• WorkflowAttributesDeleteRequest
• NamedEntityUpdateRequest

Ideally, we can start emitting some or all of these events.

Describe alternatives you've considered

NONE

Propose: Link/Inline OR Additional context

We can add more events in cloudevent_publisher.go. Then add the cloudeventPublisher.Publish(...) call to each manager like was done in execution_manager.go.

One thing to keep in mind is that the CloudEvents Spec is opinionated about what values are sent in the events such as all events from the same source should have unique IDs. An example of this challenge is the LaunchPlanUpdatedRequest which has an Id already, but the Id will be the same for all LaunchPlans.

Are you sure this issue hasn't been raised already?

• [X] Yes

Have you read the Code of Conduct?

• [X] Yes

[dosubot[bot]]

🤖

It's great to see your detailed proposal for emitting other CloudEvents in Flyte. Your thorough analysis and proposed solution are valuable contributions to the project. We encourage you to go ahead and create a pull request with your changes. Your efforts will help enhance the functionality of Flyte and benefit the entire community. Thank you for your willingness to contribute!

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.