spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-48821][SQL] Support Update in DataFrameWriterV2

Open szehon-ho opened this issue 7 months ago • 21 comments

What changes were proposed in this pull request?

Add Update support in DataFrameWriterV2

Why are the changes needed?

Spark currently supports update sql statement. We want DataFrame to have the same support.

Does this PR introduce any user-facing change?

Yes. This PR introduces new API like the following:

  spark.table("source")
    .update(Map("salary" -> lit(-1)), $"pk" >= 2)
    

How was this patch tested?

new tests

Was this patch authored or co-authored using generative AI tooling?

No

szehon-ho avatar Jul 05 '24 16:07 szehon-ho