Hyukjin Kwon comments

Results 207 comments of


                                            Hyukjin Kwon

[SPARK-40027][PYTHON][SS][DOCS] Add self-contained examples for pyspark.sql.streaming.readwriter

> either 1) you've tested these examples manually or 2) these examples will be automatically tested via CI? Both, yes :-).

[SPARK-40027][PYTHON][SS][DOCS] Add self-contained examples for pyspark.sql.streaming.readwriter

Merged to master.

[SPARK-40027][PYTHON][SS][DOCS] Add self-contained examples for pyspark.sql.streaming.readwriter

I am touching these examples a lot. So all posthoc reviews are very appreciated!

[SPARK-39832][PYTHON] Support column arguments in regexp_replace

cc @zero323 FYI

[SPARK-39832][PYTHON] Support column arguments in regexp_replace

@physinet mind enabling https://github.com/physinet/spark/actions/workflows/build_main.yml and rebasing please? Apache Spark leverages the Github resources from the PR author's fork.

[SPARK-39832][PYTHON] Support column arguments in regexp_replace

I am fine with this. I will leave it to @zero323

[SPARK-39930][SQL] Introduce Cache Hints

I like the idea.

[SPARK-39927][BUILD] Upgrade to Avro 1.11.1

Seems like the test failure looks related: ``` AvroV1Suite.support user provided avro schema for writing non-nullable enum type java.lang.AssertionError: assertion failed: Exception tree doesn't contain the expected exception of type...

[SPARK-39743][DOCS] Updated some spark.io.compression configuration descriptions to clarify parameter applica…

@zzzzming95 mind filing a JIRA please? See also https://spark.apache.org/contributing.html

[SPARK-39925][SQL] Add array_sort(column, comparator) overload to DataFrame operations

Oops, it slipped through my fingers. Mind retriggering https://github.com/brandondahler/spark/runs/7585897593?