alerting icon indicating copy to clipboard operation
alerting copied to clipboard

[FEATURE] re-model monitor executions as a cancellable task

Open eirsep opened this issue 1 year ago • 1 comments

Today Alerting execution is not tracked by Opensearch task manager because it is executed as a Job Scheduler job.

This renders us without any control over long runnign monitor exceutions which we may want to cancel or abort

All operations in Opensearch are registerable in Opensearch framework A sample task looks like

retention_lease_background_sync NS5L3EYoSM2ED7Ivqq1snQ:993186  - transport 1706578387362 01:33:07  5.1h         10.212.107.51 5112ba5b511cfd4495

Alerting monitor execution scheduled by job schuedler should register a cancellable task

alerting/monitor/scheduled NS5L3EYoSM2ED7Ivqq1snQ:993186  - transport 1706578387362 01:33:07 1.5m    10.212.107.51 5112ba5b511cfd4495

task can contain details like monitor id to help with tracking which node is running the monitor.

eirsep avatar Jan 30 '24 22:01 eirsep

We can invoke ExecuteMonitorAction in runJob flow in job scheduler

eirsep avatar Feb 09 '24 11:02 eirsep