spark-connect-rs icon indicating copy to clipboard operation
spark-connect-rs copied to clipboard

Published 20 hours ago • verified publisher icon sjrusso8

Implement `DataFrameWriterV2`

Open sjrusso8 opened this issue 10 months ago • 2 comments

Description

The initial DataFrameWriter is created but there is also another way to write data via DataFrameWriterV2. This method has a slightly different implementation and leverages a different proto command message.

I think the methods should mirror the ones found on the Spark API guide, and a new method should be added onto the DataFrame for writeTo.

sjrusso8 avatar Apr 16 '24 14:04 sjrusso8

This looks fairly straightforward, I can pick it up.

hntd187 avatar Apr 17 '24 16:04 hntd187

Sounds good! Assigned to you. Thanks!

sjrusso8 avatar Apr 17 '24 17:04 sjrusso8

Hi @hntd187 , are you currently working on this issue?

abrassel avatar May 05 '24 23:05 abrassel

Closed #30

sjrusso8 avatar May 15 '24 12:05 sjrusso8