datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Enforce the uniqueness of map key name for the map/make_map function

Open goldmedal opened this issue 1 year ago • 3 comments

Is your feature request related to a problem or challenge?

In most databases, the map type only allows unique keys. Refer to the behavior of DuckDB:

D select map {'1':1, '1':2};
Invalid Input Error: Map keys must be unique.

However, the specification of MapArray states:

/// In this layout, the keys and values are each respectively contiguous. We do
/// not constrain the key and value types, so the application is responsible
/// for ensuring that the keys are hashable and unique. Whether the keys are sorted
/// may be set in the metadata for this field.

MapArray is not responsible for ensuring the uniqueness of keys. Therefore, we should ensure the keys are unique on the application side (DataFusion).

Describe the solution you'd like

We can check the keys are distinct when invoking the map and make_map functions.

Describe alternatives you've considered

No response

Additional context

No response

goldmedal avatar Jul 12 '24 15:07 goldmedal

take

dharanad avatar Jul 12 '24 16:07 dharanad

Hi @dharanad, if you're no longer working on this ticket, I can take over.

Weijun-H avatar Aug 25 '24 02:08 Weijun-H

Hi @dharanad, if you're no longer working on this ticket, I can take over.

Hey, please feel free to take it

dharanad avatar Aug 25 '24 03:08 dharanad