[FEATURE] Add monitoring support for extensions

[saratvemulapalli]

Is your feature request related to a problem?

Plugins are embedded into OpenSearch process which prevents easy monitoring. With extensions I would love to see monitoring support which helps in a. Tracking Performance b. Tracking the progress of extension.

What solution would you like?

A simple way to measure events within the extension system and a framework to publish metrics. This is intended to be generic and we could discuss the major event types we'd like to categorize. Few thoughts (but not limited to):

• Requests to/from all extension points (helps measure against performance of plugins)
• Extension health stats like CPU, heap etc

[varuntumbe]

Hi @saratvemulapalli , @dbwiddis

Can I work on this ( if others are not planning to take this ) after I finish the current one ?

Thanks.

[saratvemulapalli]

Thanks @varuntumbe absolutely. There are lot more details involved including design, I would be glad to work with you to untangle the problem.

[varuntumbe]

Great ! @saratvemulapalli , I have got a little bit of rebasing problem going on with the current one. I will reach out to you within this week ( tomorrow or day after tomorrow ) after the completion of the current one.

Thanks.

[varuntumbe]

Hi @saratvemulapalli @dbwiddis ,

I will be free for few hours during weekend. You want me to look at any code in specific ?

Thanks and regards Varun

[varuntumbe]

Hi @saratvemulapalli ,

I will try to write one test class for initializeExtensionRequest until we finalize on the design part of this feature.

Thanks and Regards Varun

[saratvemulapalli]

Thanks @varuntumbe for picking this up. I've added discuss label, few of the requirements are added in here.

Requests to/from all extension points (helps measure against performance of plugins)

I think you can start with exploring ideas of publishing metrics for each request, we use netty underneath. Do you want to take a stab at exploring opensource solutions we could use ?

[varuntumbe]

Hi @saratvemulapalli ,

Sure let me see what I can find. Basically first thing is to track and publish the metric ( in this case requests to and from extension points ). I might have some doubts on the specificity of this feature, i will post a question as and when I get it, if it's okay.

Thanks and regards Varun

[saratvemulapalli]

@varuntumbe absolutely!!

[varuntumbe]

Hi @saratvemulapalli

I will spend some time after closing the current bug and will let you know soon

Thanks