hyperspace icon indicating copy to clipboard operation
hyperspace copied to clipboard

[FEATURE REQUEST]: Create helper function to check whether index is actually used in the plan

Open paryoja opened this issue 4 years ago • 1 comments

Feature requested

I need a function to check whether Spark actually used an index I intended to use. It could be helpful to understand the behavior of hyperspace indexes and optimize the behavior of it.

Acceptance criteria

Describe what you consider (e.g., functionality requirement) has to be true to mark this feature as complete

  • [ ] Check whether a LogicalPlan has any use of any hyperspace indexes.
  • [ ] Given hyperspace index name, check whether a LogicalPlan has a use of the index.
  • [ ] Given hyperspace index name and version value, check whether a LogicalPlan has a use of the index with the version value.

Success criteria

Define any measurements that need to be performed

  • [ ] Functionality implemented

Additional context

In the DeltaLakeIntegrationTest.scala, it has isIndexVersionUsed and isIndexUsed for checking the test results. It could be better to expose the functions for the general use.

paryoja avatar Sep 02 '21 03:09 paryoja

hs.explain(df) returns used indexes, but I think we need to improve the output result so that it shows the used log version & the number of times each index applied.

sezruby avatar Sep 02 '21 22:09 sezruby