apache
Add metrics for compactor entries read and write
#4553 says to add additional Compactor metrics, but I have to assume that we would want these for internal compactions too, right @ddanielr ?
https://github.com/apache/accumulo/issues/4553 says to add additional Compactor metrics, but I have to assume that we would want these for internal compactions too, right @ddanielr ?
I experimented with pushing the metric data all into FileCompactor in 365e26146c42d56c4d2ab66d63d48cc12142b8fb in this branch. Thinking that if the data collection is pushed there it can easily be used by tablet server and compactor. One thing I am wondering now is if the metric name should be same for both processes.
One thing I am wondering now is if the metric name should be same for both processes.
IMO, yes. The tags differentiate the process names, host names, etc.
One thing I am wondering now is if the metric name should be same for both processes.
IMO, yes. The tags differentiate the process names, host names, etc.
Are these just examples or do you think these tags should be applied here? @dlmarion
The process name and IP are set as common tags when the metrics are initialized and do not need to be added again.
Common tags cannot be added once the metrics are initialized, but you could add custom tags when you create the meter if there is a need to differentiate a metric. Adding a custom tag would be appropriate when it makes sense if multiple meters could be rolled up into and presented as a single metric.
Say something was like response.time and there were individual components that make up that value. Using a tag like "client.a", "client.b" would let the overall response.time to be presented by the metrics collection / display - and still allow the ability to drill down to the component if needed.
If they do not make sense rolled up, then using a unique name for each meter would probably be better,
I experimented with pushing the metric data all into FileCompactor in 365e261 in this branch. Thinking that if the data collection is pushed there it can easily be used by tablet server and compactor. One thing I am wondering now is if the metric name should be same for both processes.
I tried merging your branch into this one but I haven't been able to see these metrics. The new IT that I added was picking them up before but I am not seeing them at all now. Still looking into why that might be.
I tried merging your branch into this one but I haven't been able to see these metrics. The new IT that I added was picking them up before but I am not seeing them at all now. Still looking into why that might be.
@DomGarguilo just saw this comment, missed it earlier when I reviewed. I can take at a look at this.
This looks good, need to figure out those
typetags are needed.
I ended up removing the tags. It seems like the other tags that are automatically added should be good enough to not need more tags but I can add them back if anyone thinks otherwise.
I tried merging your branch into this one but I haven't been able to see these metrics. The new IT that I added was picking them up before but I am not seeing them at all now. Still looking into why that might be.
@DomGarguilo just saw this comment, missed it earlier when I reviewed. I can take at a look at this.
In dfec4a1 I made wrapper methods in Compactor for FileCompactor.getTotalEntriesWritten() and FileCompactor.getTotalEntriesRead() so that when setting up the FunctionCounter, we can use this instead of null as the object that is used. This seemed to fix the issue I was seeing where I couldnt see the metrics. Not sure if using null as the object caused the issue but thats my best guess currently. The tests passes now.