delta icon indicating copy to clipboard operation
delta copied to clipboard

[BUG] testColumnMapping("block CONVERT TO DELTA") fails with "Illegal table name" on macOS

Open jaceklaskowski opened this issue 3 years ago • 3 comments

Bug

Describe the problem

$ build/sbt 'core/testOnly *.DeltaColumnMappingSuite -- -z "block CONVERT TO DELTA"'
...
[info] - block CONVERT TO DELTA *** FAILED *** (3 seconds, 426 milliseconds)
[info]   Testing under mode: name Expected exception java.lang.UnsupportedOperationException to be thrown, but org.apache.spark.sql.catalyst.parser.ParseException was thrown (DeltaColumnMappingSuite.scala:1011)
[info]   org.scalatest.exceptions.TestFailedException:
[info]   at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
...
[info]   Cause: org.apache.spark.sql.catalyst.parser.ParseException: Illegal table name parquet./private/var/folders/k_/4tl__g7d589cw78q8f11ykrw0000gn/T/spark-9f8c3862-387f-4457-ba30-60f6d1139f90(line 1, pos 17)
[info]
[info] == SQL ==
[info] convert to delta parquet.`/private/var/folders/k_/4tl__g7d589cw78q8f11ykrw0000gn/T/spark-9f8c3862-387f-4457-ba30-60f6d1139f90` partitioned by (key1 String)
[info] -----------------^^^
[info]   at io.delta.sql.parser.DeltaSqlAstBuilder.$anonfun$visitTableIdentifier$1(DeltaSqlParser.scala:252)
[info]   at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:133)
[info]   at io.delta.sql.parser.DeltaSqlAstBuilder.visitTableIdentifier(DeltaSqlParser.scala:249)
...

Environment information

  • Delta Lake version: commit 56e1b9b417e8815cd7003eaf1796988c670b8b0d
  • macOS Big Sur 11.6.6
  • openjdk version "11.0.15" (Temurin-11.0.15+10 (build 11.0.15+10))

jaceklaskowski avatar Jul 23 '22 11:07 jaceklaskowski

delta lake only works with JDK 1.8 right ? from Readme

Under Project JDK specify a valid Java 1.8 JDK and opt to use SBT shell for project reload and builds.

or is it supposed to work with all JDK versions?

sherlockbeard avatar Jul 25 '22 17:07 sherlockbeard

I tested with both JDK 1.8 and JDK 11 and I cannot reproduce. Builds: OpenJDK Runtime Environment (build 1.8.0_322-bre_2022_02_28_15_01-b00) and OpenJDK Runtime Environment Homebrew (build 11.0.12+0). MacOs: Monterey 12.4

It's not clear for me either with what JDK versions it should work.

bogdanmrusu avatar Jul 26 '22 15:07 bogdanmrusu

@jaceklaskowski could you try again with JDK 1.8?

scottsand-db avatar Aug 02 '22 17:08 scottsand-db

Closing this as we cannot reproduce this. Feel free to re-open it if there is new update.

zsxwing avatar Sep 14 '22 17:09 zsxwing