m3
m3 copied to clipboard
Refactor ReverseMatch signature to allow for iterator reuse
This diff applies a similar approach to that found in https://github.com/m3db/m3/pull/3988 to ReverseMatch. The reasoning is the same--ReverseMatch acquires a lot of SortedTagIterators, and pooling is expensive in this code path (contention). Also, the SortedTagIterators are intentionally not closed as of now (see https://github.com/m3db/m3/blob/a873a01a6064a4b5a2538aba4421bb13b8de986c/src/x/serialize/unchecked_metric_tags_iterator.go#L68, which panics if they are), making pooling more or less useless anyhow.
Callers can now pass in a dummy SortedTagIteratorFn which actually reuses the same iterator. We may be able to clean up the interfaces further; @prateek has pointed out that the contract of SortedTagIteratorFn might be confusing now--you'd expect to be able to do:
iter1 := matchOpts.SortedTagIteratorFn(..)
iter2 := matchOpts.SortedTagIteratorFn(...)
// Use both iter1 and iter2
That won't work with iterator reuse, which isn't immediately clear from the function itself.
Alternatives considered:
- I tried just passing in a SortedTagIterator for reuse directly, but that runs into trouble with the implementation of
uncheckedMetricTagsIter, which is both a tags iterator and an ID. Calling Reset on it with the wrong data messes up the ID bytes as well. Possibly fixable, but I think better to just adopt the same approach for Forward and Reverse match.
Before (with pooling):
After (no pooling):
lgtm
correct my understanding: the final change to use this new interface is going to be in the internal repo - where we'll pass down the single iterator per batch by using your trick of returning the same object in the
matchOptions.SortedTagIteratorFn, yeah?
Yep! I'll send you the corresponding diff internally.