micrometer
micrometer copied to clipboard
micrometer-metrics
Datadog Dogstatsd support.
ref https://github.com/micrometer-metrics/micrometer-docs/pull/225 discussion
Make the smallest compatible change for users to migrate to dogstatsd approach. This still defaults to the prior API method currently. Many of our users already have the datadog-agent already installed collecting traces, logs and profiling information. This result in a positive performance and reliability benefit as your datadog-agent requires less network hops, and is configured to automatically handle edge related to retries and failure scenarios.
This allows them to update their configuration as follows:
Instead of
datadog.uri = https://api.datadoghq.com
datadog.apiKey = secret
For autodiscovery these environment variables should exist: https://github.com/DataDog/java-dogstatsd-client/blob/master/src/main/java/com/timgroup/statsd/NonBlockingStatsDClient.java#L53-L58
# note the 3rd `/` to satisfy java.net.URI
datadog.uri = discovery:///
For specifying exactly (example named pipe):
datadog.uri = unix:///var/run/datadog.sock
another example for udp to the listening port
datadog.uri = udp://localhost:8125/
@platinummonkey Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.
@shakuzen will this make it into an upcoming release? Or is there some more work left on it?
I am looking forward to this landing as my team and I are having some issues using DataDog via Micrometer in our Spring Boot application, we don't see the same issue when using java-dogstatsd-client and I would very much like us to keep using Micrometer with Spring Boot. I am thinking might just solve our problem.
The issue is that when we use Micrometer, we observe data loss. When we compare events as recorded in Splunk (our logging tool), versus Datadog, we see 2 or 4 times more events in Splunk than Datadog. And the loss factor correlates to the size of our ECS cluster. When we had our app on 4 nodes in our ECS cluster, we had 4x data loss. After scaling down to 2 nodes, we got 2x data loss. We don't see this issue when using java-dogstatsd-client. The issue might be something on our side, like bad config when using Micrormeter but so far we haven't found anything.
cc @platinummonkey
will this make it into an upcoming release? Or is there some more work left on it?
I'm particularly hoping to get feedback on this review comment. Once that is addressed, I will try to make time to try this out (and you or other Datadog users could also try it, which would be greatly appreciated). For inclusion in our upcoming 1.10 release, we are planning a release candidate on October 10. New features would need to be merged before then. Otherwise, it will have to wait until a future release.
The issue is that when we use Micrometer, we observe data loss.
I'm sorry that's happening. Feel free to open a separate issue regarding it.
Our original customer had moved off micrometer prior to this landing, but I can certainly come back and help finish this out as it’s a popular library.
Our original customer had moved off micrometer prior to this landing, but I can certainly come back and help finish this out as it’s a popular library.
Thanks @platinummonkey, that would be much appreciated!
I'm sorry that's happening. Feel free to open a separate issue regarding it.
Thanks, sounds good. We do have a support case open with folks at DataDog and they helping us look into this issue but since this isn't their officially supported mechanism, they might not be able to get to the bottom of this, we are also looking at things on our end. If we don't get a resolution from either approaches, I might just open an issue here.
the remainder of these sonatype-lift errors look related to what already exists and outside the scope of this PR 🙃
I merged main and now the tests fail
Sorry about that, main is now fixed. Could you rebase?
:warning: 11 God Classes were detected by Lift in this project. Visit the Lift web console for more details.
![sonatype-lift[bot] avatar](https://avatars.githubusercontent.com/in/9843?v=4 "Published by a pub.dev verified publisher"){kind=small}
currently seem blocked on
./gradlew resolveAndLockAll --no-build-cache --write-locks
...
* What went wrong:
Execution failed for task ':micrometer-registry-cloudwatch2:resolveAndLockAll'.
> Could not resolve all files for configuration ':micrometer-registry-cloudwatch2:compileClasspath'.
> Could not find software.amazon.awssdk:cloudwatch:.
Required by:
project :micrometer-registry-cloudwatch2
> Could not resolve software.amazon.awssdk:cloudwatch:latest.release.
Required by:
project :micrometer-registry-cloudwatch2
project :micrometer-registry-cloudwatch2 > project :micrometer-core
project :micrometer-registry-cloudwatch2 > project :micrometer-core > project :micrometer-commons
project :micrometer-registry-cloudwatch2 > project :micrometer-core > project :micrometer-observation
> Could not resolve software.amazon.awssdk:cloudwatch:2.17.293.
> Could not parse POM https://repo.maven.apache.org/maven2/software/amazon/awssdk/cloudwatch/2.17.293/cloudwatch-2.17.293.pom
> Could not find software.amazon.awssdk:services:2.17.293.
Searched in the following locations:
- https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.17.293/services-2.17.293.pom
If the artifact you are trying to retrieve can be found in the repository but without metadata in 'Maven POM' format, you need to adjust the 'metadataSources { ... }' of the repository declaration.
> Could not resolve software.amazon.awssdk:cloudwatch:2.17.293.
> Could not parse POM https://repo.maven.apache.org/maven2/software/amazon/awssdk/cloudwatch/2.17.293/cloudwatch-2.17.293.pom
> Could not find software.amazon.awssdk:services:2.17.293.
...
🛠 Lift Auto-fix
Some of the Lift findings in this PR can be automatically fixed. You can download and apply these changes in your local project directory of your branch to review the suggestions before committing.[^1]
# Download the patch
curl https://lift.sonatype.com/api/patch/github.com/micrometer-metrics/micrometer/3329.diff -o lift-autofixes.diff
# Apply the patch with git
git apply lift-autofixes.diff
# Review the changes
git diff
Want it all in a single command? Open a terminal in your project's directory and copy and paste the following command:
curl https://lift.sonatype.com/api/patch/github.com/micrometer-metrics/micrometer/3329.diff | git apply
Once you're satisfied commit and push your changes in your project. [^1]: You can preview the patch by opening the patch URL in the browser.
can we ignore the sonatype-lift here?
1 of the failures is a nag. 2 of them actually cause compile failures and are blocking.
none of the remaining sonatype failures are related to this branch, prexisting failures on main
appears another integration is failing as a result of merging main
What went wrong:
Execution failed for task ':micrometer-registry-wavefront:test'.
@platinummonkey Hoping you get this bumped. :)
The current statsd thing sends a line for every counter increase, which makes the metrics scale w/ the thing they're counting, which can be no bueno!
Hope this gets merged in soon!
Bump
Heads up @platinummonkey, the PR now looks like it acquired some merge conflicts This branch has conflicts that must be resolved Just in case they see your bump soon. :)
Heads up @platinummonkey, the PR now looks like it acquired some merge conflicts
This branch has conflicts that must be resolvedJust in case they see your bump soon. :)
this is the theme yes, when a PR sits for over a year 🙃
the .github/workflows/deploy-docs.yml workflow also seems busted now :) irrelevant to this PR