antlr4 icon indicating copy to clipboard operation
antlr4 copied to clipboard

Swift github Actions we're running python not Swift

Open parrt opened this issue 3 years ago • 48 comments

Fixes #3968.

parrt avatar Nov 19 '22 17:11 parrt

Ping @nesevis @tonyarnold

parrt avatar Nov 19 '22 18:11 parrt

Maybe @KvanTTT has an idea? Looks like this PR also fails to build.

https://github.com/antlr/antlr4/actions/runs/3504526096/jobs/5870258100

parrt avatar Nov 19 '22 18:11 parrt

Here's the relevant error:

/Users/runner/work/antlr4/antlr4: error: package at '/Users/runner/work/antlr4/antlr4' is using Swift tools version 5.6.0 but the installed version is 5.2.0

I can't do anything about it today (it's Sunday here), but I'll have a look tomorrow at work.

tonyarnold avatar Nov 19 '22 20:11 tonyarnold

Hi @tonyarnold, tried changing version but still failing. :) https://github.com/antlr/antlr4/actions/runs/3505681821/jobs/5872128382

parrt avatar Nov 20 '22 00:11 parrt

It got a little further, though, so we can start to see the underlying issues:

cpp-builds (macos-latest, gcc) is failing due to:

ccache: error: Could not find compiler "gcc-9" in PATH

build (macos-latest, swift) seems to be variations of:

/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/SwiftRunner-ForkJoinPool-1-worker-3-1668903248411/main.swift:1:8: error: no such module 'Antlr4'
import Antlr4

Let me see if I can run it locally.

tonyarnold avatar Nov 20 '22 02:11 tonyarnold

Okay, so locally it's a little clearer - there are multiple types like VisitorBasicLexer, VisitorBasicParser, etc that are used in the tests that aren't included anywhere in the repository.

Are these types supposed to be being generated prior to compilation?

tonyarnold avatar Nov 20 '22 04:11 tonyarnold

If I try to run mvn install from the project root on macOS 13.0.1 with Xcode 14.1, I get the following error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo (help-goal) on project antlr4-maven-plugin: Execution help-goal of goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo failed: Unsupported class file major version 63 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo (help-goal) on project antlr4-maven-plugin: Execution help-goal of goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo failed: Unsupported class file major version 63
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:375)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:104)
    at java.lang.reflect.Method.invoke (Method.java:578)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution help-goal of goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo failed: Unsupported class file major version 63
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:104)
    at java.lang.reflect.Method.invoke (Method.java:578)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.IllegalArgumentException: Unsupported class file major version 63
    at org.objectweb.asm.ClassReader.<init> (ClassReader.java:196)
    at org.objectweb.asm.ClassReader.<init> (ClassReader.java:177)
    at org.objectweb.asm.ClassReader.<init> (ClassReader.java:163)
    at org.objectweb.asm.ClassReader.<init> (ClassReader.java:284)
    at org.apache.maven.tools.plugin.extractor.annotations.scanner.DefaultMojoAnnotationsScanner.analyzeClassStream (DefaultMojoAnnotationsScanner.java:208)
    at org.apache.maven.tools.plugin.extractor.annotations.scanner.DefaultMojoAnnotationsScanner.scanDirectory (DefaultMojoAnnotationsScanner.java:194)
    at org.apache.maven.tools.plugin.extractor.annotations.scanner.DefaultMojoAnnotationsScanner.scan (DefaultMojoAnnotationsScanner.java:108)
    at org.apache.maven.tools.plugin.extractor.annotations.scanner.DefaultMojoAnnotationsScanner.scan (DefaultMojoAnnotationsScanner.java:84)
    at org.apache.maven.tools.plugin.extractor.annotations.JavaAnnotationsMojoDescriptorExtractor.scanAnnotations (JavaAnnotationsMojoDescriptorExtractor.java:123)
    at org.apache.maven.tools.plugin.extractor.annotations.JavaAnnotationsMojoDescriptorExtractor.execute (JavaAnnotationsMojoDescriptorExtractor.java:102)
    at org.apache.maven.tools.plugin.scanner.DefaultMojoScanner.populatePluginDescriptor (DefaultMojoScanner.java:97)
    at org.apache.maven.plugin.plugin.AbstractGeneratorMojo.execute (AbstractGeneratorMojo.java:278)
    at org.apache.maven.plugin.plugin.HelpGeneratorMojo.execute (HelpGeneratorMojo.java:95)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:104)
    at java.lang.reflect.Method.invoke (Method.java:578)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

tonyarnold avatar Nov 20 '22 04:11 tonyarnold

Hmm...i never upgrade to first OS X major release. hahah. wonder if that matters. i'm at 12.6. maven 3.8.6 already installed. mvn clean then mvn -DskipTests install.

The error Unsupported class file major version 63 seems to indicate it's using too recent of java version. mvn should be using 1.8 for runtime and 11 for tool.

                <maven.compiler.source>11</maven.compiler.source>
                <maven.compiler.target>11</maven.compiler.target>

parrt avatar Nov 20 '22 05:11 parrt

Are these types supposed to be being generated prior to compilation?

From the package description looks like they should not be included:

            exclude: [
                "./runtime/Swift/Tests/VisitorBasic.g4",
                "./runtime/Swift/Tests/VisitorCalc.g4",
                "./runtime/Swift/Tests/LexerA.g4",
                "./runtime/Swift/Tests/LexerB.g4",
                "./runtime/Swift/Tests/Threading.g4"
            ]

BUt maybe for running tests those are needed?

parrt avatar Nov 20 '22 05:11 parrt

(you're in dev branch right?)

parrt avatar Nov 20 '22 05:11 parrt

(you're in dev branch right?)

No, I've checked out a copy of the branch that this PR is representing.

Could you give me a really quick rundown of how I should be running the tests locally? It's been a decade since I've had to work with Java - my expertise is all in Apple's ecosystem these days. I'm worried that I'm doing something wrong when checking this.

tonyarnold avatar Nov 20 '22 05:11 tonyarnold

I got further after I installed a recent OpenJDK release, and added -DfailIfNoTests=false to the list of arguments being passed to mvn in the GitHub Actions workflow:

ie.

swift) mvn -X -e '-Dtest=swift.**' -DfailIfNoTests=false test ;;

That being said, it still fails. If I set my working directory to runtime/Swift and run swift test, that seems to pass now that I have the OpenJDK installed.

tonyarnold avatar Nov 20 '22 06:11 tonyarnold

Hi @tonyarnold we have some semi useful documentation here: https://github.com/antlr/antlr4/blob/master/doc/antlr-project-testing.md

Definitely have to change into the runtime test directory. I start by building the whole thing in the roof directory:

mvn -DskipTests install

then

cd runtime-testsuite/
mvn -Dtest=swift.** test

That should figure out where the appropriate run time is and run things. Yep, everything fails here... Not sure why because I could swear we had this running:

[ERROR]   SwiftRuntimeTests>RuntimeTests.lambda$runtimeTests$1:104 Test: DelegatorInvokesFirstVersionOfDelegateRule; State: Compile; java.lang.Exception: command "swift build -c release -Xswiftc -I/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -L/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -lAntlr4 -Xlinker -rpath -Xlinker /Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release"
  in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-27-1668969309765 failed
Cause:
java.lang.InterruptedException: Exit code 1 with output:
Building for production...
remark: Incremental compilation has been disabled: it is not compatible with whole module optimization[1/2] Compiling Test MBaseListener.swift
/private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-27-1668969309765/main.swift:1:8: error: no such module 'Antlr4'
import Antlr4
       ^

@KvanTTT I could swear that I saw the swift tests running a whole bunch of Go processes... Maybe the test rig is unnecessarily initializing multiple targets?

parrt avatar Nov 20 '22 18:11 parrt

actually, @KvanTTT that gives us a clue. Maybe another cut and paste error where are we are actually building Go runtime instead of Swift before running tests? No I think we are OK. Running it again it looks like it's doing a bunch of swift built. Not sure where that Go run came from in the background.

parrt avatar Nov 20 '22 18:11 parrt

If you go into RuntimeRunner.java and turn on

	public static final boolean WATCH_COMMANDS_EXEC = true;

You will see the following commands being executed. Does this help @tonyarnold ?

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.antlr.v4.test.runtime.swift.SwiftRuntimeTests
RUNNING swift build -c release in /Users/parrt/antlr/code/antlr4/runtime/Swift
RUNNING swift build -c release -Xswiftc -I/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -L/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -lAntlr4 -Xlinker -rpath -Xlinker /Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-25-1668969671413
RUNNING swift build -c release -Xswiftc -I/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -L/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -lAntlr4 -Xlinker -rpath -Xlinker /Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-3-1668969671409
RUNNING swift build -c release -Xswiftc -I/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -L/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -lAntlr4 -Xlinker -rpath -Xlinker /Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-45-1668969671417
RUNNING swift build -c release -Xswiftc -I/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -L/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -lAntlr4 -Xlinker -rpath -Xlinker /Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-39-1668969671375
...

If I run individually, I see this output:

$ swift build -c release
'antlr4': warning: Invalid Exclude '/Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/runtime/Swift/Tests/VisitorBasic.g4': File not found.
'antlr4': warning: Invalid Exclude '/Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/runtime/Swift/Tests/VisitorCalc.g4': File not found.
'antlr4': warning: Invalid Exclude '/Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/runtime/Swift/Tests/LexerB.g4': File not found.
'antlr4': warning: Invalid Exclude '/Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/runtime/Swift/Tests/LexerA.g4': File not found.
'antlr4': warning: Invalid Exclude '/Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/runtime/Swift/Tests/Threading.g4': File not found.
'antlr4': warning: found 5 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/LexerB.g4
    /Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/VisitorBasic.g4
    /Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/Threading.g4
    /Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/VisitorCalc.g4
    /Users/parrt/antlr/code/antlr4/runtime/Swift/Tests/Antlr4Tests/LexerA.g4

Building for production...
/Users/parrt/antlr/code/antlr4/runtime/Swift/Sources/Antlr4/UnbufferedCharStream.swift:336:9: warning: switch covers known cases, but 'Stream.Status' may have additional unknown values, possibly added in future versions
        switch stream.streamStatus {
        ^
/Users/parrt/antlr/code/antlr4/runtime/Swift/Sources/Antlr4/UnbufferedCharStream.swift:336:9: note: handle unknown values using "@unknown default"
        switch stream.streamStatus {
        ^
[2/2] Archiving libAntlr4Static.a
Build complete! (11.01s)

It looks like it is building libraries up at the root:

beast:dev:~/antlr/code/antlr4/runtime-testsuite $ find .. -name libAntlr4Static.a
../.build/x86_64-apple-macosx/release/libAntlr4Static.a
../.build/arm64-apple-macosx/release/libAntlr4Static.a
../.build/arm64-apple-macosx/debug/libAntlr4Static.a

OK so the file not found issue is easy to fix... Just change Package.swift. I just push the new commit for that so you can do a pull. Build is a lot cleaner now:

beast:dev:~/antlr/code/antlr4/runtime-testsuite $ swift build -c release
Building for production...
/Users/parrt/antlr/code/antlr4/runtime/Swift/Sources/Antlr4/UnbufferedCharStream.swift:336:9: warning: switch covers known cases, but 'Stream.Status' may have additional unknown values, possibly added in future versions
        switch stream.streamStatus {
        ^
/Users/parrt/antlr/code/antlr4/runtime/Swift/Sources/Antlr4/UnbufferedCharStream.swift:336:9: note: handle unknown values using "@unknown default"
        switch stream.streamStatus {
        ^
[2/2] Archiving libAntlr4Static.a
Build complete! (10.37s)

parrt avatar Nov 20 '22 18:11 parrt

The key issue here is that this directory does not exist:

/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release

I think I just have to fix the runner to use the correct directory.

parrt avatar Nov 20 '22 18:11 parrt

pulling in dev branch into this branch... Seems some stuff was not pulled in here. Retrying everything.

parrt avatar Nov 20 '22 18:11 parrt

Wow. Is this really the right command? Seems very repetitive:

swift build -c release -Xswiftc -I/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -L/Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release -Xlinker -lAntlr4 -Xlinker -rpath -Xlinker /Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release

Anyway from a test directories I get

/private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-3-1668971122442/main.swift:1:8: error: module 'Antlr4' was created for incompatible target x86_64-apple-macosx10.10: /Users/parrt/antlr/code/antlr4/runtime/Swift/.build/release/Antlr4.swiftmodule
import Antlr4
       ^

parrt avatar Nov 20 '22 19:11 parrt

Hey! I believe the Package.Swift file needs to live in the root directory of the repository for Swift Package Manager to 'see' the package: https://github.com/antlr/antlr4/commit/e96de77feb9e383c5f81dc50ed3e240c979af128

nesevis avatar Nov 20 '22 19:11 nesevis

oh crap. ok, reverting. thanks.

parrt avatar Nov 20 '22 19:11 parrt

There's definitely something wrong with the way we are running test because they refer to .build in runtime/Swift but the build directories actually up at the root.

parrt avatar Nov 20 '22 19:11 parrt

@nesevis any chance you could figure out what's the right command line option should be to run the tests? The relevant file is runtime-testsuite/test/org/antlr/v4/test/runtime/swift/SwiftRunner.java

parrt avatar Nov 20 '22 19:11 parrt

After moving the file back to the root position:

beast:/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-3-1668971122442 $ swift build -c release -Xswiftc -I/Users/parrt/antlr/code/antlr4/.build/release -Xlinker -L/Users/parrt/antlr/code/antlr4/.build/release -Xlinker -lAntlr4 -Xlinker -rpath -Xlinker /Users/parrt/antlr/code/antlr4/.build/release
Building for production...
/private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-3-1668971122442/MParser.swift:168:19: warning: left side of nil coalescing operator '??' has non-optional type 'String', so the right side is never used
                        print(("S.a") ?? "nil")
                              ~~~~~~~ ^~~~~~~~
                                      
error: link command failed with exit code 1 (use -v to see invocation)
ld: library not found for -lAntlr4
[1/2] Linking Test
beast:/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-3-1668971122442 $ ls -l /Users/parrt/antlr/code/antlr4/.build/release
lrwxr-xr-x  1 parrt  staff  26 Nov 20 11:21 /Users/parrt/antlr/code/antlr4/.build/release@ -> arm64-apple-macosx/release

I changed the directories in SwiftRunner. At least now the directories are correct but it still feels to find Antlr4 lib during linking.

parrt avatar Nov 20 '22 19:11 parrt

ok, i'm giving up... Hopefully somebody else can fix this.

parrt avatar Nov 20 '22 19:11 parrt

ah. a clue. When I run from intellij, @KvanTTT, the build is for x86 despite being on an M1 mac:

/Users/parrt/antlr/code/antlr4/.build/release@ -> x86_64-apple-macosx/release

when I build with plain swift build -c release from the command line it makes the correct release: release@ -> arm64-apple-macosx/release.

Yes I have confirmed that when you use the Java ProcessBuilder you get the wrong architecture when running the same command. It seems to build for x86 where is from the command line the exact same command uses arm64. There must be something in the environment for ProcessBuilder, but I'm not sure what it is.

AH! It must be something in the PATH that is using a different version of Swift because I get a warning when running from the command line that I do not get when I run from inside intellij / java.:

/Users/parrt/antlr/code/antlr4/runtime/Swift/Sources/Antlr4/UnbufferedCharStream.swift:336:9: warning: switch covers known cases, but 'Stream.Status' may have additional unknown values, possibly added in future versions
        switch stream.streamStatus {
        ^
/Users/parrt/antlr/code/antlr4/runtime/Swift/Sources/Antlr4/UnbufferedCharStream.swift:336:9: note: handle unknown values using "@unknown default"
        switch stream.streamStatus {
        ^

The command line says

$ swift --version
swift-driver version: 1.45.2 Apple Swift version 5.6.1 (swiftlang-5.6.0.323.66 clang-1316.0.20.12)
Target: arm64-apple-macosx12.0

Running from java says:

swift-driver version: 1.45.2 Apple Swift version 5.6.1 (swiftlang-5.6.0.323.66 clang-1316.0.20.12)
Target: x86_64-apple-macosx12.0

WTF?

parrt avatar Nov 20 '22 19:11 parrt

ah! My java was x86. damn. ok, so now it is building arm64 properly in .build. :) still getting the same error is however

Building for production...
remark: Incremental compilation has been disabled: it is not compatible with whole module optimization[1/2] Compiling Test TBaseListener.swift
/private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-23-1668975009692/TParser.swift:132:44: warning: left side of nil coalescing operator '??' has non-optional type 'String', so the right side is never used
                        print((_localctx.r.toStringTree(self)) ?? "nil")
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^~~~~~~~
                                                               
/private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-23-1668975009692/TParser.swift:239:7: warning: variable '_prevctx' was written to, but never read
                var _prevctx: EContext = _localctx
                    ^
error: link command failed with exit code 1 (use -v to see invocation)
ld: library not found for -lAntlr4
[1/2] Linking Test

parrt avatar Nov 20 '22 20:11 parrt

@parrt try updating the Package.swift to look like this:

// swift-tools-version:5.6

import PackageDescription

let package = Package(
    name: "Antlr4",
    products: [
        .library(
            name: "Antlr4",
            targets: ["Antlr4"]),
        .library(
            name: "Antlr4Static",
            type: .static,
            targets: ["Antlr4"]),
    ],
    targets: [
        .target(
            name: "Antlr4",
            dependencies: [],
            path: "./runtime/Swift/Sources/Antlr4"),
        .testTarget(
            name: "Antlr4Tests",
            dependencies: ["Antlr4"],
            path: "./runtime/Swift/Tests/Antlr4Tests",
            exclude: [
                "gen/LexerB.tokens",
                "gen/VisitorCalcLexer.tokens",
                "gen/VisitorBasicLexer.tokens",
                "gen/VisitorBasic.tokens",
                "gen/Threading.interp",
                "gen/VisitorCalcLexer.interp",
                "gen/ThreadingLexer.interp",
                "gen/VisitorBasicLexer.interp",
                "gen/LexerB.interp",
                "gen/Threading.tokens",
                "gen/VisitorCalc.interp",
                "gen/LexerA.interp",
                "gen/ThreadingLexer.tokens",
                "gen/LexerA.tokens",
                "gen/VisitorCalc.tokens",
                "gen/VisitorBasic.interp",
                "VisitorBasic.g4",
                "VisitorCalc.g4",
                "LexerA.g4",
                "LexerB.g4",
                "Threading.g4"
            ]
        )
    ]
)

That compiles and tests pass for me on macOS 13.0.1 with Xcode 14.1.

tonyarnold avatar Nov 21 '22 04:11 tonyarnold

org.opentest4j.AssertionFailedError: Test: LRWithLabels; State: Compile; java.lang.Exception: command "swift build -c release -Xswiftc -I/Users/parrt/antlr/code/antlr4/.build/release -Xlinker -L/Users/parrt/antlr/code/antlr4/.build/release -Xlinker -lAntlr4 -Xlinker -rpath -Xlinker /Users/parrt/antlr/code/antlr4/.build/release"
  in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-23-1669064769634 failed
Cause:
java.lang.InterruptedException: Exit code 1 with output:
Building for production...
remark: Incremental compilation has been disabled: it is not compatible with whole module optimization[1/2] Compiling Test TBaseListener.swift
/private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-23-1669064769634/TParser.swift:132:44: warning: left side of nil coalescing operator '??' has non-optional type 'String', so the right side is never used
                        print((_localctx.r.toStringTree(self)) ?? "nil")
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^~~~~~~~
                                                               
/private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-23-1669064769634/TParser.swift:239:7: warning: variable '_prevctx' was written to, but never read
                var _prevctx: EContext = _localctx
                    ^
error: link command failed with exit code 1 (use -v to see invocation)
ld: library not found for -lAntlr4
[1/2] Linking Test

parrt avatar Nov 21 '22 21:11 parrt

This should be simple: ld: library not found for -lAntlr4 must be just an -I or -L issue right?

parrt avatar Nov 21 '22 21:11 parrt

This should be simple […]

😅

Yeah, it should.

I've tried a bunch to run the same script that is being used by GitHub Actions (mvn -X -e '-Dtest=swift.**' -DfailIfNoTests=false test), however I cannot get past this problem:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo (help-goal) on project antlr4-maven-plugin: Execution help-goal of goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo failed: Unsupported class file major version 63 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo (help-goal) on project antlr4-maven-plugin: Execution help-goal of goal org.apache.maven.plugins:maven-plugin-plugin:3.6.2:helpmojo failed: Unsupported class file major version 63

I've tried both OpenJDK 18 & 19, and have installed Maven 3.8.6. I'm willing and able to help with the Swift problems, but I'm not sure how to resolve the Java issues that I'm seeing.

tonyarnold avatar Nov 21 '22 22:11 tonyarnold