spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-40027][PYTHON][SS][DOCS] Add self-contained examples for pyspark.sql.streaming.readwriter

Open HyukjinKwon opened this issue 2 years ago • 1 comments

What changes were proposed in this pull request?

This PR proposes to improve the examples in pyspark.sql.streaming.readwriter by making each example self-contained with a brief explanation and a bit more realistic example.

Why are the changes needed?

To make the documentation more readable and able to copy and paste directly in PySpark shell.

Does this PR introduce any user-facing change?

Yes, it changes the documentation

How was this patch tested?

Manually ran each doctest.

HyukjinKwon avatar Aug 10 '22 06:08 HyukjinKwon

cc @viirya and @HeartSaVioR mind taking a look please when you find some time?

HyukjinKwon avatar Aug 10 '22 06:08 HyukjinKwon

either 1) you've tested these examples manually or 2) these examples will be automatically tested via CI?

Both, yes :-).

HyukjinKwon avatar Aug 11 '22 01:08 HyukjinKwon

Merged to master.

HyukjinKwon avatar Aug 11 '22 11:08 HyukjinKwon

I am touching these examples a lot. So all posthoc reviews are very appreciated!

HyukjinKwon avatar Aug 11 '22 11:08 HyukjinKwon