datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Add support for `CopyTo::partition_by` field in datafusion_proto

Open alamb opened this issue 1 year ago • 4 comments

After https://github.com/apache/arrow-datafusion/pull/9240 datafusion protobuf will will silently ignore partition_by options;

The task is

  1. add code to serialize / deserialize the CopyTo::partition_by field, added in https://github.com/apache/arrow-datafusion/pull/9240 to the existing serialization code (similarly to how copy_to is done) https://github.com/apache/arrow-datafusion/blob/f1dcbf978210be6aa69b07892cf47321905aeece/datafusion/proto/src/logical_plan/mod.rs#L850

  2. Update the test in datafusion/proto/tests/cases/roundtrip_logical_plan.rs to send a non empty partition column list

Originally posted by @devinjdangelo in https://github.com/apache/arrow-datafusion/pull/9240#discussion_r1491159044

alamb avatar Feb 16 '24 14:02 alamb

Once https://github.com/apache/arrow-datafusion/pull/9240 is merged, I think this would be a good first issue as it will be small and self contained

alamb avatar Feb 16 '24 14:02 alamb

I can work on this one.

PhVHoang avatar Feb 16 '24 15:02 PhVHoang

I can work on this one.

Thanks @PhVHoang 🙏

https://github.com/apache/arrow-datafusion/pull/9240 was just merged so I think this one is now actionable

alamb avatar Feb 19 '24 07:02 alamb

take

PhVHoang avatar Feb 20 '24 06:02 PhVHoang