clickhouse-java
clickhouse-java copied to clipboard
Syntax error when use spark jdbc catalog
trafficstars
Describe the bug
The same sql but not get same result between spark jdbc catalog and DBeave(by jdbc connection)
Steps to reproduce
spark = SparkSession.builder \
.config("spark.jars", "https://repo1.maven.org/maven2/com/clickhouse/clickhouse-jdbc/0.3.2-patch11/clickhouse-jdbc-0.3.2-patch11-all.jar") \
.appName("JDBCTableCatalogExample") \
.getOrCreate()
spark.conf.set("spark.sql.catalog.xx","org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog")
spark.conf.set("spark.sql.catalog.xx.url","jdbc:clickhouse://xxx")
spark.conf.set("spark.sql.catalog.xx.driver","com.clickhouse.jdbc.ClickHouseDriver")
spark.conf.set("spark.sql.catalog.xx.user","xxx")
spark.conf.set("spark.sql.catalog.xx.password","xxx")
df = spark.sql("SELECT * FROM xx.xx_db.aa t final")
df.show()
Expected behaviour
sql could run and return result
Code example
Error log
File "/Users/tianci.yu/Library/Python/3.8/lib/python/site-packages/pyspark/sql/utils.py", line 117, in deco raise converted from None pyspark.sql.utils.ParseException: extraneous input 'final' expecting {<EOF>, ';'}(line 1, pos 105) == SQL == SELECT * FROM xx.xx_db.aa t final
if i removed alais "t" ,the sql became "SELECT * FROM xx.xx_db.aa final", then it run normal
Why do you expect SparkSQL syntax to recognise Clicksouse's FINAL keyword?