[RFC]: Metrics Refactoring #1492 Draft PR

[joshuaan7]

Fixes #1492

TorchServe defines metrics in a metrics.yaml file, including both frontend metrics (i.e. ts_metrics) and backend metrics (i.e. model_metrics). When TorchServe is started, the metrics definition is loaded in frontend and backend cache separately. Backend flushes metrics cache once a load model or inference request is done.

Type of change

• [x] New feature (non-breaking change which adds functionality)

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced. Please also list any relevant details for your test configuration.

Checklist:

• [ x] Did you have fun?
• [ x] Have you added tests that prove your fix is effective or that this feature works?
• [ x] Has code been commented, particularly in hard-to-understand areas?
• [ x] Have you made corresponding changes to the documentation?

[joshuaan7]

@lxning @maaquib @msaroufim Hi all, I am opening this PR up for review. I'm not sure if I have permissions to add reviewers but if there are more reviewers that should be added, please feel free to add. Apologies in advance for the large PR

[maaquib]

Closing in favour of #1954