Query Cost Model and Usage

[atris]

This JIRA tracks building a mechanism for identifying and computing the indicative cost of a query, based on parameters such as query complexity and data statistics (cardinality, selectivity etc).

The cost model will be pluggable -- you just need to implement the interfaces.

The cost can be used for stuff like throttling and cancelling of expensive queries in the face of limited resource utilisation.

In a later stage, we can explore using the cost for a more intelligent query optimiser.

Note that this is more of a "logical" cost -- using statistics and query cyclomatic complexity.

Related work: https://github.com/apache/pinot/issues/9055

[atris]

cc @yupeng9 @Jackie-Jiang

[amrishlal]

This should be very useful. Are we planning to build a stats / metadata store as part of this or will the cost model be applied on top of individual queries only without the use of stats store?