bzhao

[SPARK-39942][PYTHON][PS] Need to verify the input nums is integer in nsmallest func

12

The input parameter of nsmallest should be validated as Integer. So I think we might miss this validation. And PySpark will raise Error when we input the strange types into...

[SPARK-39938][PYTHON][PS] Accept all inputs of prefix/suffix which implement __str__ in add_predix/add_suffix

2

We need to follow the pandas behavior of prefix/suffix parameter validation in add_prefix/add_suffix. Now, we force to validate it as a String type. But pandas looks all values which can...

[SPARK-39941][PYTHON][PS] window and min_periods should be integer in rolling func

2

window and min_periods parameters is not be validated in rolling function. ### What changes were proposed in this pull request? Validate the said 2 parameters to be a integer only...

[SPARK-39821][PYTHON][PS] Fix error during using DatetimeIndex

11

Pandas disallow conversion between datetime/timedelta and conversions for any datetimelike to float. This will raise error in PYSPARK, during we simply call a DatetimeIndex. So we need to avoid to...

[SPARK-39822][PYTHON][PS] Provide a good feedback to users

6

Provide a graceful error msg to users when they build Index with different dtypes. ### What changes were proposed in this pull request? Raise a graceful error when users create...

[SPARK-39824][PYTHON][PS] Introduce index where and putmask func in pyspark

12

### What changes were proposed in this pull request? Add more pyspark pandas Index func which is similar with pandas. ### Why are the changes needed? Add where and putmask...

[SPARK-39654][PYTHON][PS] Parameters quotechar and escapechar need limitation in read_csv

1

Currently, as doc description of pyspark pandas and pandas itself, these 2 options are only support 1-char input, so let's make pyspark pandas follow this behavior. Issue affected: https://issues.apache.org/jira/browse/SPARK-39654 ###...

[ARM CI] openlab is EOL

1

So apologized that openLab was EOL recently. We are no choice and forced to close openlab services. That means the original regression test on openLab ARM machines are dead forever....

[DISCUSS] hyperscan support ARM

34

Hi hyperscan team, I'm an newbee for hyperscan project. I'm so excited to have a conversation with you. We have a plan to make hyperscan to support ARM64 function. And...

[FLINK-28825][k8s] Add K8S pod scheduler into Kubernetes options

2

We introduce 3 options for supporting specify the K8S pod scheduler name. ## What is the purpose of the change *(For example: This pull request makes task deployment go through...