Latest MLton fails to build on 10.6–10.7 (only for x86_64)

[barracuda156]

@MatthewFluet Here are results on buildbots: https://ports.macports.org/port/mlton/details We got two systems failing: 10.6 and 10.7, and on 10.6 only x86_64 build fails. (On 10.6 both ppc and i386 are fine.) I guess, MLton 20180207 is just too new, I should fallback to an earlier version for 10.6–10.7 x86_64?

Here are relevant logs: https://build.macports.org/builders/ports-10.6_x86_64-builder/builds/162718/steps/install-port/logs/stdio https://build.macports.org/builders/ports-10.7_x86_64-builder/builds/152346/steps/install-port/logs/stdio

Both segfault on the same spot, though 10.6 sets arch to x86:

/opt/local/bin/gsed -e "s/MLTON_NAME/MLton/" -e "s/MLTON_VERSION/"20230721"/" < control/version_sml.src > control/version.sml
/opt/local/bin/gsed -e "s/MLTON_DEBUG/false/" -e "s/HOST_ARCH/x86/" -e "s/HOST_OS/darwin/" < ../lib/stubs/mlton-stubs/pre-mlton_sml.src > ../lib/stubs/mlton-stubs/pre-mlton.sml
chmod -w front-end/mlb.grm.sml
(cat ../lib/stubs/mlton-stubs/pre-mlton_sml.src; echo 'false' 'x86' 'darwin'; cat ../lib/stubs/mlton-stubs/pre-mlton.sml) | openssl sha1 | /opt/local/bin/gsed 's/.*\([a-z0-9]\{40\}\).*/\1/' > ../lib/stubs/mlton-stubs/pre-mlton_sml.chk
(cat control/version_sml.src; echo 'MLton' '"20230721"'; cat control/version.sml) | openssl sha1 | /opt/local/bin/gsed 's/.*\([a-z0-9]\{40\}\).*/\1/' > control/version_sml.chk
/bin/sh: line 1: 66483 Segmentation fault      /opt/local/libexec/mlton-bootstrap/bin/mlton -stop f "$1"
/bin/sh: line 1: 66544 Segmentation fault      /opt/local/libexec/mlton-bootstrap/bin/mlton -stop f "$1"
/bin/sh: line 1: 66551 Segmentation fault      /opt/local/libexec/mlton-bootstrap/bin/mlton -stop f "$1"
make[2]: *** [mlton-stubs.mlb] Error 1
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_mlton/mlton/work/mlton-63666ac8ab65da81b9ec9595e7053d26e0682f51/mlton'
make[1]: *** [compiler] Error 2

While 10.7 sets it to arm64:

/opt/local/bin/gsed -e "s/MLTON_NAME/MLton/" -e "s/MLTON_VERSION/"20230721"/" < control/version_sml.src > control/version.sml
/opt/local/bin/gsed -e "s/MLTON_DEBUG/false/" -e "s/HOST_ARCH/amd64/" -e "s/HOST_OS/darwin/" < ../lib/stubs/mlton-stubs/pre-mlton_sml.src > ../lib/stubs/mlton-stubs/pre-mlton.sml
cp -fpR front-end/mlb.grm.sml.boot front-end/mlb.grm.sml
chmod -w front-end/ml.grm.sig
chmod -w front-end/mlb.grm.sig
chmod -w front-end/mlb.grm.sml
(cat ../lib/stubs/mlton-stubs/pre-mlton_sml.src; echo 'false' 'amd64' 'darwin'; cat ../lib/stubs/mlton-stubs/pre-mlton.sml) | openssl sha1 | /opt/local/bin/gsed 's/.*\([a-z0-9]\{40\}\).*/\1/' > ../lib/stubs/mlton-stubs/pre-mlton_sml.chk
(cat control/version_sml.src; echo 'MLton' '"20230721"'; cat control/version.sml) | openssl sha1 | /opt/local/bin/gsed 's/.*\([a-z0-9]\{40\}\).*/\1/' > control/version_sml.chk
/bin/sh: line 1: 33626 Segmentation fault: 11  /opt/local/libexec/mlton-bootstrap/bin/mlton -stop f "$1"
/bin/sh: line 1: 33683 Segmentation fault: 11  /opt/local/libexec/mlton-bootstrap/bin/mlton -stop f "$1"
/bin/sh: line 1: 33690 Segmentation fault: 11  /opt/local/libexec/mlton-bootstrap/bin/mlton -stop f "$1"
make[2]: *** [mlton-stubs.mlb] Error 1
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_mlton/mlton/work/mlton-63666ac8ab65da81b9ec9595e7053d26e0682f51/mlton'
make[1]: *** [compiler] Error 2
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_mlton/mlton/work/mlton-63666ac8ab65da81b9ec9595e7053d26e0682f51'
make: *** [all] Error 2

No idea why arch is picked differently, identical MLton is used for bootstrap for both. Clang differs though: 10.6 has Clang-11, 10.7 has Clang-15. (This we can change as we like.)

Details of bootstrap MLton choice are here: https://github.com/macports/macports-ports/blob/master/lang/mlton-bootstrap/Portfile

[MatthewFluet]

The choice of HOST_ARCH is determined by ./bin/platform, which essentially just queries uname -m. I suspect the difference between 10.6 and 10.7 is just various points in macOS evolution --- the set of architectures supported vs the default architecture reported.

As to the segmentation fault, it could also be that MLton 20180207 is too old -- there have at various times been compiler bugs that make it impossible to use an older binary on a newer platform, such as alignment assumptions that were technically invalid, but tolerated, on older platforms that are now strictly enforced on newer platforms.

[barracuda156]

The point is that 20180207 works fine for 10.8–10.12 on x86_64, but fails on 10.6–10.7 x86_64, while 20100608 works fine for i386 on 10.6.

But I guess, I should just try that out and see which version works for old x86_64. I have a local setup of 10.6 on Intel, I just happened to test 10.8 for Intel (which worked).

[MatthewFluet]

As far as I can tell from the logs, this issue is mistitled. It isn't really the case that "Latest MLton fails to build on 10.6-10.7 (only for x86_64)". The real issues is that the chosen bootstrap binaries segfault on those systems. As suggested, one could go back to older releases. I'm not sure that there is anything that can be done here (i.e., with the current MLton sources) to resolve this issue.

[MatthewFluet]

One option, for ${configure.build_arch} eq "x86_64" and ${os.major} <= 16, might be to use a binary x86 release (e.g., 20100608), because there may not be any older amd64-darwin releases of MLton.

[barracuda156]

@MatthewFluet Thank you, you are right re title, but I wasn’t initially sure of the cause and wanted to confirm.

Will address this on the weekend. (Not a difficult fix, but needs a bit more than throwing in an extra version for bootstrap: the code fixing libgmp paths currently assumes that those differ by arch. Since 2013 and 2010 have different paths in pre-built binaries from later ones, x86_64 case has to be modified to handle old Macports and old Homebrew bootstrap binaries both.)

[MatthewFluet]

Closing; as noted in the comments, this is a boostrapping problem and not something that can be resolved here (i.e., with the current MLton sources.