spark-rapids icon indicating copy to clipboard operation
spark-rapids copied to clipboard

[FEA] Support querying map scalars with key vectors

Open mythrocks opened this issue 3 years ago • 0 comments

Followup to #6288.

The spark-rapids plugin currently supports querying map scalars only with key scalars. Map vectors can be queried with both scalar and vector keys. So the following query falls back to CPU:

SELECT MAP( 1, 'one', 2, 'two', 3, 'three' )[ my_key_column ] FROM my_table;

This might require the addition of a map_scalar_view in CUDF JNI, sharing a back-end with maps_column_view. Another approach (Thanks, @revans2!) might be to unroll the map scalar into an if-else. This might be feasible since the map can't be too massive, given that it's specified as a constant in the query.

mythrocks avatar Aug 11 '22 17:08 mythrocks