micrometer icon indicating copy to clipboard operation
micrometer copied to clipboard

Publish relative to PushMeterRegistry initialization time and align StepMeter boundaries to that

Open lenin-jaganathan opened this issue 3 years ago • 9 comments

This PR tries to solve this issue (https://github.com/micrometer-metrics/micrometer/issues/2818). The start time is captured when the push meter registry is initialized. For Step based meters, this is passed to the Step Value and StepTupple to align them to the registry start time and not the step start time (see - https://github.com/micrometer-metrics/micrometer/issues/1218).

TODO:

  • [ ] rebase and target 1.8.x
  • [ ] investigate test failures with the alignToEpoch config flipped

lenin-jaganathan avatar Oct 04 '22 08:10 lenin-jaganathan

@lenin-jaganathan Please sign the Contributor License Agreement!

Click here to manually synchronize the status of this Pull Request.

See the FAQ for frequently asked questions.

pivotal-cla avatar Oct 04 '22 08:10 pivotal-cla

@lenin-jaganathan Thank you for signing the Contributor License Agreement!

pivotal-cla avatar Oct 04 '22 08:10 pivotal-cla

:warning: 10 God Classes were detected by Lift in this project. Visit the Lift web console for more details.

sonatype-lift[bot] avatar Oct 04 '22 08:10 sonatype-lift[bot]

@lenin-jaganathan can you change this PR to draft mode?

jamesmartinpp avatar Oct 06 '22 16:10 jamesmartinpp

can you change this PR to draft mode?

@jamesmartinpp Why do you think that should be done? I'm in the process of reviewing it now.

shakuzen avatar Oct 07 '22 01:10 shakuzen

@shakuzen it doesn't really matter. I thought draft mode might be safer. I'm ok proceeding with the PR as is.

jamesmartinpp avatar Oct 07 '22 15:10 jamesmartinpp

After flipping the default, there is a test failure. I've run out of time to dig into it, and I'll be on vacation next week. I can come back to this once I'm back from vacation, but feel free to look into the failure in the meantime.

shakuzen avatar Oct 07 '22 18:10 shakuzen

@shakuzen If the default behavior is made to publish relative to the start time, a few tests will fail because of the way they were written. I will try to address those tests.

lenin-jaganathan avatar Oct 08 '22 11:10 lenin-jaganathan

@shakuzen Addressed the test failures in SignalFx library.

lenin-jaganathan avatar Oct 18 '22 12:10 lenin-jaganathan

Added an additional commit to make the metrics timestamp synchronized across instances(report timestamp as the start of the current step) for SignalFx Meter Registry.

lenin-jaganathan avatar Oct 31 '22 09:10 lenin-jaganathan

@shakuzen Do you want me to address anything more on this PR?

lenin-jaganathan avatar Nov 16 '22 10:11 lenin-jaganathan

Do you want me to address anything more on this PR?

I don't think so. I'm asking for some review internally. Once that is done, we should be good to go. If you have any thoughts on naming as I've recently updated, please share. Or if you have thoughts on the below.

On how we'll apply this where, we'll keep the default for the configuration false when merging this to main for the next feature release (1.11), which makes the publishing happen not at the same time for every instance. Out of an abundance of caution, now I would like to introduce this in patch releases (1.8.x, 1.9.x, 1.10.x) but initially with the default to true so there is no behavior change unless a user overrides the default. If we get some feedback on usage of this in production, we could consider flipping the default in a patch release later, as the current behavior is very problematic in large deployments. Until we get some feedback on usage, I'm not comfortable making the change in a patch release, even with an option for users to opt out of it. Another thing we're considering is making the configuration option deprecated in 1.11 because as long as there is no problem with the behavior, I'd rather not have a configuration toggle for it going forward (we didn't have one until now because we wanted consistent behavior and there wasn't a need for users to be able to choose). At some point in the future, we could consider removing the configuration altogether.

shakuzen avatar Nov 17 '22 07:11 shakuzen

Putting this in draft mode. Will close this PR in a week. For additional info see the linked issue for why this fix might not be desired.

lenin-jaganathan avatar Apr 06 '23 03:04 lenin-jaganathan

Fixed by https://github.com/micrometer-metrics/micrometer/pull/3750

lenin-jaganathan avatar Apr 11 '23 04:04 lenin-jaganathan