druid icon indicating copy to clipboard operation
druid copied to clipboard

Correct aggregators violating names

Open sreemanamala opened this issue 1 year ago • 2 comments

Description

In case of few aggregators for example BloomSqlAggregator, BaseVarianceSqlAggregator etc, the aggName is being updated from a0 to a0:agg, breaching the contract as we would expect the aggName as the name which is passed. This is causing a mismatch while creating a column accessor.

This commit aims to correct those violating sql aggregators.


Key changed/added classes in this PR
  • CompressedBigDecimalSqlAggregatorBase
  • TDigestGenerateSketchSqlAggregator
  • DoublesSketchObjectSqlAggregator
  • BloomFilterSqlAggregator
  • BaseVarianceSqlAggregator

This PR has:

  • [x] been self-reviewed.
    • [ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
  • [ ] added documentation for new or modified features or behaviors.
  • [ ] a release note entry in the PR description.
  • [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • [ ] added or updated version, license, or notice information in licenses.yaml
  • [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • [ ] added integration tests.
  • [ ] been tested in a test Druid cluster.

sreemanamala avatar Jun 17 '24 02:06 sreemanamala

I think that change is the finalization stuff in action; how the finalization happens after these changes?

I'm not sure if forcing the column name would be the best - as for some reason the original aggfactory have decided to add a postagg - we can't really ignore that....if the base aggregate factory is not prepared to do the finalization implicitly then I guess that might go south...so I think we should probably look for some alternate path:

  • fix finalization support correctly
    • prefferably by moving the finalization to a separate project or something
    • create an autofinalized cover for the aggregates -
  • make it clear that these aggregators are not supported in windowing

kgyrtkirk avatar Jun 17 '24 12:06 kgyrtkirk

@kgyrtkirk can you please review the comments ? Thank you

pranavbhole avatar Jun 25 '24 04:06 pranavbhole