spark

spark copied to clipboard

[SPARK-48652][SQL] Fix casting issue in Spark SQL when comparing string column to integer value

5

### What changes were proposed in this pull request? This PR addresses a type casting issue in Spark SQL where comparing a string column to an integer value results in...

Zawa-ll

### What changes were proposed in this pull request? Introducing two new error classes instead of _LEGACY_ERROR_TEMP_2000. Classes introduced: - DATETIME_ARGUMENT_OUT_OF_RANGE - ILLEGAL_INTERVAL_ARGUMENT_VALUE ### Why are the changes needed? We...

mihailom-db

[SPARK-49491][SQL] Replace AnyRefMap with HashMap

19

### What changes were proposed in this pull request? Replace AnyRefMap with HashMap ### Why are the changes needed? HashMap has better performance in Scala 2.13: https://issues.apache.org/jira/browse/SPARK-49491 ### Does this...

George314159

[SPARK-49162][SQL] Push down date_trunc function

3

### What changes were proposed in this pull request? This PR: - Adds support for DATE_TRUNC in V2 optimization pushdown - Consumes this pushdown for Postgres & H2 Connectors ###...

IvanK-db

[SPARK-49893] Respect user schema nullability for file data sources when DSV2 Table is used.

9

### What changes were proposed in this pull request? DataFrameReader has 3 APIs for JSON reading json(DataSet[String]) json(Rdd) json(filePath) First two APIs respects provided user schema nullability when spark flag...

urosstan-db

[SPARK-49730][SQL] classify syntax errors for pgsql, mysql, sqlserver and h2

2

### What changes were proposed in this pull request? In this PR I propose that we add classification of JDBC driver exception that represent syntax errors on external databases. Queries...

ivanjevtic-db

[SPARK-49768][SQL] Provide error conditions for `make_date`/`make_timestamp` errors `_LEGACY_ERROR_TEMP_2000`

4

### What changes were proposed in this pull request? This PR proposes to provide proper error conditions for `_LEGACY_ERROR_TEMP_2000` and improve the error message ### Why are the changes needed?...

itholic

### What changes were proposed in this pull request? This PR adds support to define event time column in the output dataset of `TransformWithStateInPandas` operator. The new event time column...

jingz-db

[SPARK-49695][SQL] Postgres fix xor push-down

4

### What changes were proposed in this pull request? This PR fixes the pushdown of ^ operator (XOR operator) for Postgres. Those two databases use this as exponent, rather then...

andrej-db