Assembly produces broken JAR file on case-insensitive file systems due to overlapping class names in ECMA 2020 parser

[x5g]

Hello, I ran the following command, ./ostrich tests/case-insensitive-2.smt2

but I got an unexpected result. Why? (error "ecma2020regex/Absyn/ControlLettera")

Did I do something wrong?

[pruemmer]

Can you give us some information about your platform; and did you compile Ostrich yourself, or did you use the provided binary?

We have seen this problem before on some systems with case-insensitive file system, but have not been able to hunt down the cause of the problem yet.

[x5g]

Hello, my laptop is MacOS 12.3.1. Xcode is not installed, but sbt 1.5.5, Scala 2.13.7 and Java 8 are installed (and I seem to see that Java 17 is installed during the installation of sbt. I'm not sure if it is really installed). I have compiled myself. The following is the information after I execute sbt assembly.

[info] welcome to sbt 1.5.5 (Homebrew Java 17.0.1) [info] loading settings for project ostrich-master-build from assembly.sbt ... and [info] Passed: Total 236, Failed 0, Errors 0, Passed 236

So, I think I compiled it successfully. I executed ./ostrich ./tests/parse-regex2.smt2 and I got sat

But as mentioned above, I found that when the SMT file contains re.from_ecma2020, and the regular expression contains letters, there will be unexpected output. For example, ./ostrich ./tests/parse-ecma-bug1.smt2 -> sat, but ./ostrich ./tests/parse-ecma-bug2.smt2 -> (error "ecma2020regex/Absyn/ControlLettera").

If there is no effective solution at present, can you provide some information about the system environment that can run ostrich successfully? Thanks a lot.

[amandasystems]

This is almost certainly the problem with building on a case-insensitive file system, which is the default on macOS and windows. AFAICT this is not a problem with ostrich specifically but rather a bug or possibly miscondifuration in Scala and/or SBT assembly, but we have not had the time to reproduce the exact conditions and report a bug upstream. At the very least sbt should give an error when producing code that won’t run, and certainly not give runtime errors like this.

To test if this is indeed your problem, you can try compiling ostrich on an external drive or in a volume that’s formatted with a case sensitive file system. I’ll post instructions for how to do this later if you don’t already know how to do that. The process is similar but opposite to how people make apps that require a case-insensitive file system run on a case-sensitive one.

[amandasystems]

Ok, here's the steps to work around this bug (if that is what you are affected by).

1. Open Disk Utility
2. ⌘N (or New Image -> Blank Image)
3. Under Format: choose "APFS (Case-sensitive)"
4. Give it a name, I assume you picked Ostrich
5. Enter a reasonable size (I choose 1 GB. Note that this is the maximum since the image is sparse -- it will only occupy as much space as you use)
6. Go to wherever you saved your image and mount it by double-clicking it
7. The disk is in /Volumes/TheNameYouPicked, in my case Ostrich
8. (Optional) verify that it's working:

$ cd /Volumes/Ostrich
$ touch ostrich
$ touch Ostrich
$ ll
Permissions Size User   Date Modified Name
.rw-r--r--     0 amanda 21 Apr 09:04  ostrich
.rw-r--r--     0 amanda 21 Apr 09:04  Ostrich

9. Check out and build Ostrich as normal in this folder

Note that the target JAR will run on any machine with a JVM, including one with a case-insensitive file system. It's only during the build process that something goes wrong.

[amandasystems]

Ok, I have isolated the problem (ping @pruemmer). If I extract the ecma2020-regex jar file I find this:

$ unzip ecma2020-regex-parser.jar
$ cd ecma2020regex
$ fd ".*ControlLetter(a|A).*"
Absyn/ControlLetterA.class
Absyn/ControlLettera.class

One of these classes is clearly lost when unzipping the jar file during SBT assembly to create the new jar file, when they touch the case-insensitive file system. Either the "rename" merge strategy for sbt assembly will work, or sbt assembly has a very serious problem.

Will investigate this later.

[amandasystems]

Update: I timed out trying to figure out how to reconfigure SBT assembly.

[x5g]

Thank you for your answer. It solved my problem perfectly!

[pruemmer]

Can you give us some information about your platform; and did you compile Ostrich yourself, or did you use the provided binary?

We have seen this problem before on some systems with case-insensitive file system, but have not been able to hunt down the cause of the problem yet.