Inconsistent handling of metrics

[Voxelot]

We currently have a cli arg to enable metrics. However, there are many inconsistencies with the metrics system overall:

• Several places always log metrics even if the flag is disabled
• Some places record metrics but never encode them into the response
• Our devops config doesn't even enable the metrics flag, since the key metrics we need aren't respecting the flag. But this means there should be additional metrics in our deployment envs that we are missing.

The goal of this ticket is to normalize all the metrics, ensuring that they are all properly recorded and made available only when the cli arg is enabled. And that no metrics are recorded (saving runtime overhead) if the cli arg is disabled.

[MkDev11]

It appears that you have a ticket or task related to addressing inconsistencies in your metrics system. To achieve the goal of normalizing metrics and ensuring they are properly recorded and made available only when a specific command-line argument is enabled, you can follow these steps:

1. Review Existing Code: Start by reviewing the existing codebase to identify all the places where metrics are being recorded or logged. This includes both the places where metrics are logged regardless of the flag state and where metrics are recorded but not properly encoded into the response.

2. Update Metric Recording Logic: Modify the code at these identified places to respect the metrics flag. Ensure that metrics are only recorded and logged when the flag is enabled. You may need to wrap metric recording logic in conditional statements that check the flag's status.

3. Encode Metrics in Responses: For the places where metrics are recorded but not encoded into the response, update the code to include the metrics in the response data when the flag is enabled. This ensures that the metrics are available for analysis.

4. Flag Configuration: Review your CLI argument or configuration for enabling/disabling metrics. Ensure that it is properly implemented and that it can be easily toggled during runtime.

5. Testing: Thoroughly test the changes to ensure that metrics are behaving as expected. Verify that metrics are recorded and logged only when the flag is enabled and that they are included in responses as needed.

6. Documentation: Update documentation or comments in the codebase to reflect the changes made to the metrics system and how it can be configured using the CLI argument.

7. DevOps Configuration: If your DevOps configuration doesn't enable the metrics flag, update it to reflect the changes and ensure that the key metrics are properly configured for deployment environments.

8. Deployment: Once you are confident in the changes, deploy the updated code to your production or deployment environments.

9. Monitoring and Maintenance: Continuously monitor the metrics system to ensure that it remains consistent and functions as expected. Address any issues or discrepancies promptly.

10. Feedback and Iteration: Gather feedback from your team and users regarding the updated metrics system. Be prepared to make further improvements based on their feedback and any new requirements that arise.

By following these steps, you can standardize your metrics system, ensure that it respects the flag for enabling/disabling metrics, and provide a more consistent and reliable monitoring solution for your application.