Hao Zhu
Hao Zhu
I wish we can support `json_tuple`. ``` from pyspark.sql.functions import * data = [("1", '''{"f1": "value1", "f2": "value2"}'''), ("2", '''{"f1": "value12"}''')] df = spark.createDataFrame(data, ("key", "jstring")) df.select(df.key, json_tuple(df.jstring, 'f1', 'f2')).collect()...
I wish whenever using Databricks + Spark RAPIDS, users do not need to specify any extra Spark or Spark RAPIDS config in the "Spark config" section. Could we set below...
I wish we can support parse_url function. eg: ``` spark-sql> select parse_url(c_customer_id,'HOST') from tpcds.customer limit 10; ! parse_url(c_customer_id#1, HOST, false) cannot run on GPU because GPU does not currently support...
I wish we can support function array_distinct. Eg: ``` from pyspark.sql.functions import * df = spark.createDataFrame([(["a", "b", "a"], ["b", "c"]), (["a","a"], ["b", "c"]), (["aa"], ["b", "c"]) ], ['x', 'y']) df.write.format("parquet").mode("overwrite").save("/tmp/testparquet")...
On ubuntu 20 dataproc 2.1(At least this is what I have tested so far), the `apt-get update` failed with: ``` apt-get update E: Repository 'https://packages.cloud.google.com/apt google-cloud-logging-focal-all InRelease' changed its 'Codename'...
**Is your feature request related to a problem? Please describe.** I wish we can support to_utc_timestamp for US/Central timezone. Such as ``` to_utc_timestamp(tz, 'US/Central') ``` This might be a sub-task...
I wish we can support `randn` function. The reason is when i am trying to generate some sample data, it is pretty useful. However currently it is falling back to...
I wish we can support `sha2` function. eg in spark-sql: ``` select sha2(c_customer_id,256) from tpcds.customer limit 3; ``` Not-supported-messages: ``` ! sha2(cast(c_customer_id#3 as binary), 256) cannot run on GPU because...