ANTLR4 doesn't generate correct code to Java grammar

[fabriciofx]

Hi!

I'm using ANTLR 4.10.1 do parse Java grammar to golang code but it doesn't convert all code necessary to compile it correctly. Look at the image:

The command I've usedto generate code: java -jar antlr-4.10.1-complete.jar -o parser -Dlanguage=Go {JavaLexer,JavaParser}.g4)

Where I got the grammar: https://github.com/antlr/grammars-v4/tree/master/java/java

[kaby76]

Antlr 4.10.1 is used in tests in CI Github Actions in grammars-v4, and the java/java/ grammar works fine in Go--although it is very slow for one of the tests.

$ git clone https://github.com/antlr/grammars-v4.git
Cloning into 'grammars-v4'...
remote: Enumerating objects: 36131, done.
remote: Counting objects: 100% (874/874), done.
remote: Compressing objects: 100% (599/599), done.
remote: Total 36131 (delta 237), reused 808 (delta 211), pack-reused 35257
Receiving objects: 100% (36131/36131), 38.69 MiB | 25.53 MiB/s, done.
Resolving deltas: 100% (18834/18834), done.
Updating files: 100% (6738/6738), done.

Kenne@DESKTOP-DL44R7B MINGW64 ~/issue-3751
$ cd grammars-v4/java/java

Kenne@DESKTOP-DL44R7B MINGW64 ~/issue-3751/grammars-v4/java/java
$ trgen -t Go
C:/msys64/home/Kenne/issue-3751/grammars-v4/java/java/
Rendering template file from Go/antlr_resource/case_changing_stream.go to Generated/antlr_resource/case_changing_stream.go
Rendering template file from Go/go.mod to Generated/go.mod
Rendering template file from Go/makefile to Generated/makefile
Rendering template file from Go/Test.go to Generated/Test.go
Rendering template file from Go/test.sh to Generated/test.sh
Rendering template file from Go/tester.psm1 to Generated/tester.psm1
Copying source file from C:/msys64/home/Kenne/issue-3751/grammars-v4/java/java/README.md to Generated/README.md
Copying source file from C:/msys64/home/Kenne/issue-3751/grammars-v4/java/java/pom.xml to Generated/pom.xml
Copying source file from C:/msys64/home/Kenne/issue-3751/grammars-v4/java/java/JavaParser.g4 to Generated/parser/JavaParser.g4
Copying source file from C:/msys64/home/Kenne/issue-3751/grammars-v4/java/java/JavaLexer.g4 to Generated/parser/JavaLexer.g4
Copying source file from C:/msys64/home/Kenne/issue-3751/grammars-v4/java/java/Benchmarks.md to Generated/Benchmarks.md

Kenne@DESKTOP-DL44R7B MINGW64 ~/issue-3751/grammars-v4/java/java
$ cd Generated/

Kenne@DESKTOP-DL44R7B MINGW64 ~/issue-3751/grammars-v4/java/java/Generated
$ make
export | grep GO
export GOPATH="C:\\Users\\kenne\\go"
export LOGONSERVER="\\\\DESKTOP-DL44R7B"
go env
set GO111MODULE=off
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Kenne\AppData\Local\go-build
set GOENV=C:\Users\Kenne\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\kenne\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\kenne\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.17.2
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\msys64\tmp\go-build1873489428=/tmp/go-build -gno-record-gcc-switches
go version
go version go1.17.2 windows/amd64
export GO111MODULE=on; go get github.com/antlr/antlr4/runtime/Go/[email protected]
go get: added github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20220411172017-c7567ba9bb40
java -jar C:/Users/Kenne/Downloads/antlr4-4.10.1-complete.jar -encoding utf-8 -Dlanguage=Go -o parser -lib parser -package parser  parser/JavaLexer.g4
java -jar C:/Users/Kenne/Downloads/antlr4-4.10.1-complete.jar -encoding utf-8 -Dlanguage=Go -o parser -lib parser -package parser  parser/JavaParser.g4
export GO111MODULE=on; go build Test.go

Kenne@DESKTOP-DL44R7B MINGW64 ~/issue-3751/grammars-v4/java/java/Generated
$ make test
bash test.sh
../examples/AllInOne11.java
Time: 0.128 s
Parse succeeded.
../examples/AllInOne17.java
Time: 0.310 s
Parse succeeded.
../examples/AllInOne7.java
Time: 0.142 s
Parse succeeded.
../examples/AllInOne8.java
Time: 0.193 s
Parse succeeded.
../examples/ManyStringsConcat.java
Time: 38.651 s
Parse succeeded.
../examples/module-info.java
Time: 0.003 s
Parse succeeded.

Kenne@DESKTOP-DL44R7B MINGW64 ~/issue-3751/grammars-v4/java/java/Generated
$ go version
go version go1.17.2 windows/amd64

[fabriciofx]

Hi @kaby76 !

I think you're using a third-party application (trgen), I'm right? I haven't found anything about it in ANTLR official documentation, for this reason I haven't get the same result as you. Which third-party application have you used? How can I install it? Can you give more information about it?

But, is there any way to do tha same thing whithout third-party applications? Just the downloaded complete jar file int the official site?

Best Regards!

[kaby76]

trgen is an application that is part of my grammar toolkit based on Antlr. There isn't any other "create your driver program for the given grammar from template" anywhere else. But I do have some canned "create the Arithmentic example with driver from template" over in Antlr4Templates, just none for Go yet.

Your Go program should be modeled after this: Generated.zip

Maybe the doc for Go could be improved. For example, it doesn't say so, but you really should place the grammar and the generated files in a directory called parser/, because the tool itself can never generate a parser without a package. By default, antlr4 -Dlanguage=Go MyGrammar.g4, is in "." and contains "package parser" at the top of the generated file. Go language requires the code to go into a directory called "parser/".

[fabriciofx]

@kaby76 thanks for send generated files, but it's important to my projects build them from scratch. So, I'm trying install yout Dometech.Trash files, but I'm getting an error. I'll open an issue in Dometech.Trash, where I think more appropriate, ok?

[fabriciofx]

trgen is an application that is part of my grammar toolkit based on Antlr. There isn't any other "create your driver program for the given grammar from template" anywhere else. But I > Maybe the doc for Go could be improved. For example, it doesn't say so, but you really should place the grammar and the generated files in a directory called parser/, because the tool itself can never generate a parser without a package. By default, antlr4 -Dlanguage=Go MyGrammar.g4, is in "." and contains "package parser" at the top of the generated file. Go language requires the code to go into a directory called "parser/".

I've moved the generated files to parser directory, but the problem is the missing code which doesn't generated. Can you test if the code is correctly generated only using the antlr jar file and the *.g4 grammar files?