volcano icon indicating copy to clipboard operation
volcano copied to clipboard

Add support for webhook notifications when vcjob finishes in different states (completed/failed/aborted/terminated).

Open chuqing1997 opened this issue 9 months ago • 2 comments

What is the problem you're trying to solve

Summary

Add support for webhook notifications when vcjob finishes in different states (completed/failed/aborted/terminated).

Motivation

Users need to be notified when their jobs complete, fail, or are terminated. Currently, there's no built-in mechanism to trigger external actions (like HTTP requests or MQ messages) when job lifecycle events occur.

Proposed Solution

  1. Add new Events in volcano-sh/apis:

    • JobCompleted
    • JobAborted
    • JobTerminated
    • JobFailed
  2. Add new Action in volcano-sh/apis:

    • WebHookAction
  3. Implement webhook functionality in volcano controller to handle WebHookAction

Use Cases

  • Send notifications to external monitoring systems
  • Trigger downstream workflows
  • Update external databases when jobs complete
  • Send alerts when jobs fail

Implementation Plan

  • [ ] Add Events and Action definitions in apis repo
  • [ ] Implement webhook logic in volcano controller
  • [ ] Add configuration for webhook URLs and methods
  • [ ] Add documentation and examples

Implementation Plan

Describe the solution you'd like

Please take a look at my description of the problem and if there are any issues with the solution. If not, I will try to complete this feature and submit a PR.

Additional context

No response

chuqing1997 avatar Jun 09 '25 12:06 chuqing1997

It's reasonable, many users need it, are you willing to contribute it?

JesseStutler avatar Jun 09 '25 13:06 JesseStutler

Yes, I'm willing to try to complete this feature, I'll do my best.

---- Replied Message ---- | From | Jesse @.> | | Date | 06/09/2025 21:01 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [volcano-sh/volcano] Add support for webhook notifications when vcjob finishes in different states (completed/failed/aborted/terminated). (Issue #4356) | JesseStutler left a comment (volcano-sh/volcano#4356)

It's reasonable, many users need it, are you willing to contribute it?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

chuqing1997 avatar Jun 09 '25 14:06 chuqing1997

/assign

chuqing1997 avatar Jun 21 '25 10:06 chuqing1997

/cc

Monokaix avatar Jun 23 '25 08:06 Monokaix

The purpose of this requirement is to add four Events: JobTerminatedEvent, JobCompletedEvent, JobAbortedEvent, and JobFailed Event, as well as one WebhookAction Action, which users can configure in policies..

I am currently working on some development in the repository after my fork. Below are my main tasks, and I also have some questions. Please help me take a look at any suggestions and areas for improvement.

The work I have already done:

  1. I have added the triggering of events in the state of pkg/controllers/job/state (when the state changes, I send event requests to the API to generate job completed/failed/bordered/terminated events, etc.);
  2. Added WebHookExecutor (pkg/controllers/job/webhook/executor. go), where the State under pkg/controllers/job/state calls the Execute method with a new check for whether it is a WebHookAction. If it is, the WebHook will be executed;
  3. Added mappings for events and actions in pkg/webhooks/delegation/jobs/validate/til.go;

My doubts:

  1. I am wondering if Webhook Action should be designed to be configurable only for jobs, or can tasks also be configured? If only the job can be configured, do I need to modify the validatePolicys method in pkg/webhooks/delegation/jobs/validate/til.go to implement it?
  2. Continuing from the previous question, if both the job and task support WebhookAction, in order to indicate which job or task it is from in the request body of the wehhook, you need to modify the pkg/controllers/job/job_comtroller_util.go-GetStateAction method to support all actions to include Target information (currently only RestartTaskAction and RestartPodAction have Target information).

@JesseStutler

chuqing1997 avatar Jun 27 '25 06:06 chuqing1997