ldc Apple Silicon (macOS + arm64)

[x] druntime unittests => all druntime tests pass in both arm64 and x86_64 on the DTK
[x] that last phobos test => one test fail in both arm64 and x86_64 on the DTK (EDIT: not related to this task)
[x] test C++ integration
[x] test Obj-C integration
[x] check arm64 ABI with linking to C (~~EDIT: pretty sure there at least one ABI problem with C linkage~~ EDIT2 => actually no C ABI problem found)
[x] new Obj-C test => all work

The new Mac support both x86_64 and arm64 architectures at once.

//------------ main.d --------------
void main(string[] args)
{
}

On Apple Silicon (Big Sur beta), you can built it with

% ldc2 main.d

which builds a x86_64 executable! It is all emulated, including all of Phobos etc. Great because existing products work.

However if you want something native and faster, no such luck:

% ldc2 -march arm64 main.d
ld: warning: ignoring file main.o, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: /usr/bin/cc failed with status: 1

The linker is assuming x86_64 as the output arch. Maybe it assumes the iPhone emulator is the target?

We have a test kit if you need testing done, and can also sponsor compatibility work (in addition of our monthly sponsorship).

Sep 11 '20 20:09 p0nce

Going further, if we force ld to output arm64:

% ldc2 main.d -march arm64 -L=-arch -L=arm64 
ld: warning: ignoring file /Users/ponce/Desktop/compilers/ldc2-1.22.0-osx-x86_64/lib/libphobos2-ldc.a, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/ponce/Desktop/compilers/ldc2-1.22.0-osx-x86_64/lib/libdruntime-ldc.a, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
Undefined symbols for architecture arm64:
  "__d_dso_registry", referenced from:
      _ldc.register_dso in main.o
  "__d_run_main", referenced from:
      _main in main.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

So maybe we could reuse the iOS lib-phobos and lib-runtime that comes builtin in LDC ?

% ldc2 main.d -mtriple arm64-apple-ios -L=-arch -L=arm64
clang: warning: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk' [-Wmissing-sysroot]
ld: library not found for -lSystem
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: /usr/bin/cc failed with status: 1

So it's where I am, not sure where to go next.

Sep 11 '20 21:09 p0nce

Consider the following main.d:

void main(string[]args)
{
	import std.stdio;
	writeln("hello Big Sur");
}

In ldc2.conf, cheat in the following way. We just reuse stdlib built for iOS.

"arm64-apple-macos":
{
    switches = [
        "-defaultlib=phobos2-ldc,druntime-ldc",
        "-link-defaultlib-shared=false",
        "-fvisibility=hidden",
        "-Xcc=-target",
        "-Xcc=arm64-apple-macos11.0",
        "-Xcc=-mmacos-version-min=11.0",
        "-Xcc=-isysroot",
        "-Xcc=/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk",
    ];

    lib-dirs = [
        "%%ldcbinarypath%%/../lib-ios-arm64",
    ];
    rpath = "%%ldcbinarypath%%/../lib-ios-arm64";
};

This create lots of warnings:

% ldc2 main.d -mtriple arm64-apple-macos
ld: warning: building for macOS, but linking in object file (xxxx.o) built for iOS. // dozens of times

% ./main 
hello Big Sur

Ultimately it seems to work.

% file main
main: Mach-O 64-bit executable arm64

Sep 11 '20 21:09 p0nce

Building the proper druntime/Phobos libs should be possible with ldc-build-runtime for the time being - unless some AArch64-specific stuff is wrongly guarded by version (iOS) etc. See https://github.com/ldc-developers/ldc/blob/4ec8fc008974b12564c0f347f5c530049bfcdd17/.azure-pipelines/posix.yml#L278-L287 So adding prebuilt macOS arm64 libs should be a tiny CI addition and doesn't need extra sponsoring, thx anyway. :)

[Wrt. -march for LDC itself, if I could I would just get rid of it - I cannot recommend -mtriple enough over that option.]

Sep 11 '20 21:09 kinke

Noted.

I'll let you know how the runtime building goes.

The good news is that on the DUB side there isn't too much to do, just doing dub test -a arm64-apple-macos works. I'm able to run unittests for intel-intrinsics, with stack traces and runtime etc.

Sep 11 '20 21:09 p0nce

A bit chicken and egg: to build runtime for arm64-apple-macos you need a private SDK /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk, not sure when it will be distributed publicly.

Sep 11 '20 21:09 p0nce

We'll need that Xcode/SDK to be available for the Azure VM, is that already covered with the v12 beta? See https://github.com/microsoft/azure-pipelines-image-generation/blob/master/images/macos/macos-10.15-Readme.md#xcode.

Sep 11 '20 21:09 kinke

Yes, with strong probability the macOS 11.0 SDK is available in this beta.

Sep 11 '20 21:09 p0nce

Re. building the runtime: I got the furthest with the following cmdline:

./ldc-build-runtime -j8 --dflags="-mtriple=arm64-apple-macos" --cflags="--target=arm64-apple-macos" --linkerFlags="-arch;arm64"

(Note: I'm using LDC 1.22)

(Other flag combinations have conflicts between arch decided by LDC / clang and the linker, so stuff end up being discarded by the linker.)

With the right flags, compilation halts on:

Undefined symbols for architecture arm64:
  "___fpclassify", referenced from:
      __D4core4stdc4math10fpclassifyFNaNbNiNeeZi in math.o
  "___isfinite", referenced from:
      __D4core4stdc4math8isfiniteFNaNbNiNeeZi in math.o
  "___isinf", referenced from:
      __D4core4stdc4math5isinfFNaNbNiNeeZi in math.o
  "___isnan", referenced from:
      __D4core4stdc4math5isnanFNaNbNiNeeZi in math.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Sep 11 '20 22:09 p0nce

That should be because of https://github.com/ldc-developers/druntime/blob/dad4d08fded06cdb51cd6fd9388407c61f78d1e3/src/core/stdc/math.d#L1035-L1056. The version (OSX) block should most likely be removed, as IIRC we require 10.9 for running prebuilt LDC itself anyway.

Sep 11 '20 22:09 kinke

Yes, and there are a few other places like this in druntime, special-casing older OSX. Going to sleep, thanks for your help! (and it's not always clear when the new symbols appear in the OS X timeline, tricky?)

Sep 11 '20 22:09 p0nce

We can use @weak to provide fallback implementations for older versions.

Sep 11 '20 22:09 kinke

In any case I'll need to cross-check between x86_64 and arm64, maybe the symbols could be different. (EDIT: yes, they are)

Sep 12 '20 09:09 p0nce

We'll need that Xcode/SDK to be available for the Azure VM, is that already covered with the v12 beta?

Yes, it is. Well, actually, for the first beta there were two downloads, one with and one without support for the new target macOS Arm64.

Sep 15 '20 14:09 jacob-carlborg

The version (OSX) block should most likely be removed, as IIRC we require 10.9 for running prebuilt LDC itself anyway.

Feel free to drop support for 10.7.

Sep 15 '20 14:09 jacob-carlborg

If you need any help, feel free to reach out.

Sep 15 '20 14:09 jacob-carlborg

It's everything related to shared libraries that makes me anxious @jacob-carlborg. What problems can you foresee?

Sep 16 '20 09:09 p0nce

It's everything related to shared libraries that makes me anxious @jacob-carlborg.

Everything? That's not very specific 😃. What exactly are you thinking of?

What problems can you foresee?

The only thing I can think of right now that could cause problems is the new cache for the dynamic libraries. But that's more related to Big Sur than to ARM64.

Sep 16 '20 14:09 jacob-carlborg

I know there might be other problem related to ARM64 on Apple platforms, both iOS and macOS:

The page size is different on Apple ARM64 and x86-64. There are a lot of hardcoded occurrences of the value 4096 in druntime and/or Phobos. I've fixed the ones that caused the unit tests to fail. But a lot of them remain. Some might cause problems, some will probably not.
Spin locks should be avoided. I don't know if that's being used

Sep 16 '20 14:09 jacob-carlborg

(Workaround not being ideal: using the druntime built for iOS as a shortcut doesn't cut it in general, due to the mostly disabled std.process)

Sep 30 '20 23:09 p0nce

So what's the status? Invoking ldc-build-runtime with --testrunners (or ninja all-test-runners if you've switched to proper manual building) builds the druntime/Phobos unittest runners and can be used to check the status quo (incl. linker errors for missing referenced symbols). Without BUILD_SHARED_LIBS=OFF, these should include shared versions of these testrunners (tiny executables linked against the shared unittest libs IIRC), which should be good enough for testing basic shared libs support.

Oct 01 '20 00:10 kinke

I was working on something else until now (porting software to ARM) and will get into the meat of the runtime on the DTK very soon. Thanks for advice.

Oct 01 '20 07:10 p0nce

Got the symbol issues solved (EDIT: not yet), now the unittests are not all passing.

0.004s PASS debug64 std.algorithm.comparison
0.005s PASS debug64 std.algorithm.iteration
0.095s PASS debug64 std.algorithm.mutation
0.007s PASS debug64 std.algorithm.searching
0.213s PASS debug64 std.algorithm.setops
0.437s PASS debug64 std.algorithm.sorting
0.011s PASS debug64 std.array
0.013s PASS debug64 std.ascii
0.001s PASS debug64 std.base64
0.006s PASS debug64 std.bigint
0.005s PASS debug64 std.bitmanip
0.000s PASS debug64 std.complex
0.001s PASS debug64 std.concurrency
0.001s PASS debug64 std.container.array
0.000s PASS debug64 std.container.binaryheap
0.000s PASS debug64 std.container.dlist
0.000s PASS debug64 std.container
0.015s PASS debug64 std.container.rbtree
0.000s PASS debug64 std.container.slist
0.001s PASS debug64 std.container.util
0.010s PASS debug64 std.conv
0.000s PASS debug64 std.csv
0.084s PASS debug64 std.datetime.date
0.002s PASS debug64 std.datetime.interval
0.282s PASS debug64 std.datetime
0.004s PASS debug64 std.datetime.stopwatch
1.181s PASS debug64 std.datetime.systime
0.103s PASS debug64 std.datetime.timezone
0.000s PASS debug64 std.demangle
0.008s PASS debug64 std.digest.crc
0.000s PASS debug64 std.digest.hmac
0.039s PASS debug64 std.digest.md
0.000s PASS debug64 std.digest.murmurhash
0.108s PASS debug64 std.digest
0.086s PASS debug64 std.digest.ripemd
0.461s PASS debug64 std.digest.sha
0.011s PASS debug64 std.encoding
0.004s PASS debug64 std.exception
0.001s PASS debug64 std.experimental.allocator.building_blocks.affix_allocator
0.001s PASS debug64 std.experimental.allocator.building_blocks.aligned_block_list
****** FAIL debug64 std.experimental.allocator.building_blocks.allocator_list
core.exception.AssertError@std/experimental/allocator/building_blocks/allocator_list.d(861): Assertion failure
----------------
/Users/ponce/Desktop/compilers/ldc2-1.24.0-beta1-osx-x86_64/bin/ldc-build-runtime.tmp/ldc-src/runtime/druntime/src/core/internal/array/equality.d:35 onAssertError [0x1062549c3]
variant.d:840 _d_assert [0x106254ef3]
/Users/ponce/Desktop/compilers/ldc2-1.24.0-beta1-osx-x86_64/bin/ldc-build-runtime.tmp/ldc-src/runtime/phobos/std/range/package.d:10803 void std.experimental.allocator.building_blocks.allocator_list.__unittest_L821_C9() [0x10536590b]
/Users/ponce/Desktop/compilers/ldc2-1.24.0-beta1-osx-x86_64/bin/ldc-build-runtime.tmp/ldc-src/runtime/phobos/std/experimental/allocator/common.d:361 std.experimental.allocator.building_blocks.allocator_list.__unittest [0x105384b5b]
test_runner.d:73 void test_runner.doTest(object.ModuleInfo*, ref core.runtime.UnitTestResult) [0x104544503]
test_runner.d:57 int test_runner.testAll().__foreachbody1(object.ModuleInfo*) [0x10454465f]
conv.d:0 int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0x1062827a3]
minfo.d:777 int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0x10629c323]
sections_darwin_64.d:204 int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0x10629ebcb]
minfo.d:770 int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0x10629c28b]
/Users/ponce/Desktop/compilers/ldc2-1.24.0-beta1-osx-x86_64/bin/ldc-build-runtime.tmp/ldc-src/runtime/druntime/src/core/internal/array/equality.d:22 int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0x106280a1b]
test_runner.d:55 core.runtime.UnitTestResult test_runner.testAll() [0x104544453]
test_runner.d:25 core.runtime.UnitTestResult test_runner.tester() [0x104544153]
variant.d:262 runModuleUnitTests [0x10625f4af]
format.d:0 void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x1062903b3]
conv.d:150 void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x106290323]
variant.d:342 _d_run_main2 [0x106290197]
variant.d:0 _d_run_main [0x10628feb7]
test_runner.d:99 main [0x104544737]
0.254s PASS debug64 std.experimental.allocator.building_blocks.ascending_page_allocator
0.010s PASS debug64 std.experimental.allocator.building_blocks.bitmapped_block
0.000s PASS debug64 std.experimental.allocator.building_blocks.bucketizer
0.004s PASS debug64 std.experimental.allocator.building_blocks.fallback_allocator
0.000s PASS debug64 std.experimental.allocator.building_blocks.free_list
0.000s PASS debug64 std.experimental.allocator.building_blocks.free_tree
0.002s PASS debug64 std.experimental.allocator.building_blocks.kernighan_ritchie
0.000s PASS debug64 std.experimental.allocator.building_blocks.null_allocator
0.000s PASS debug64 std.experimental.allocator.building_blocks.quantizer
0.017s PASS debug64 std.experimental.allocator.building_blocks.region
0.000s PASS debug64 std.experimental.allocator.building_blocks.scoped_allocator
0.003s PASS debug64 std.experimental.allocator.building_blocks.segregator
0.001s PASS debug64 std.experimental.allocator.building_blocks.stats_collector
0.000s PASS debug64 std.experimental.allocator.common
0.000s PASS debug64 std.experimental.allocator.gc_allocator
0.000s PASS debug64 std.experimental.allocator.mallocator
0.000s PASS debug64 std.experimental.allocator.mmap_allocator
0.027s PASS debug64 std.experimental.allocator
0.000s PASS debug64 std.experimental.allocator.showcase
0.000s PASS debug64 std.experimental.allocator.typed
Overflow on binary operator: int(-2147483648) / const(int)(-1)
Overflow on binary operator: int(-2147483648) * const(int)(-1)
Overflow on binary operator: int(-2147483648) / const(int)(-1)
0.000s PASS debug64 std.experimental.checkedint
0.047s PASS debug64 std.experimental.logger.core
0.001s PASS debug64 std.experimental.logger.filelogger
0.000s PASS debug64 std.experimental.logger.multilogger
0.000s PASS debug64 std.experimental.logger.nulllogger
0.000s PASS debug64 std.experimental.typecons
0.019s PASS debug64 std.file
0.006s PASS debug64 std.format
0.004s PASS debug64 std.functional
0.001s PASS debug64 std.getopt
0.000s PASS debug64 std.internal.cstring
0.000s PASS debug64 std.internal.math.biguintcore
0.000s PASS debug64 std.internal.math.biguintnoasm
0.000s PASS debug64 std.internal.math.errorfunction
0.000s PASS debug64 std.internal.math.gammafunction
0.000s PASS debug64 std.internal.scopebuffer
0.000s PASS debug64 std.internal.test.dummyrange
0.000s PASS debug64 std.internal.test.range
0.000s PASS debug64 std.json
0.004s PASS debug64 std.math
0.000s PASS debug64 std.mathspecial
0.004s PASS debug64 std.meta
0.000s PASS debug64 std.mmfile
0.038s PASS debug64 std.net.curl
0.004s PASS debug64 std.net.isemail
0.001s PASS debug64 std.numeric
0.000s PASS debug64 std.outbuffer
0.000s PASS debug64 std
0.846s PASS debug64 std.parallelism
0.063s PASS debug64 std.path
stderr test, please ignore
5.139s PASS debug64 std.process
1.605s PASS debug64 std.random
0.000s PASS debug64 std.range.interfaces
0.013s PASS debug64 std.range
0.004s PASS debug64 std.range.primitives
0.018s PASS debug64 std.regex.internal.generator
0.000s PASS debug64 std.regex.internal.ir
0.001s PASS debug64 std.regex.internal.kickstart
0.004s PASS debug64 std.regex.internal.parser
0.270s PASS debug64 std.regex.internal.tests
0.037s PASS debug64 std.regex.internal.tests2
0.007s PASS debug64 std.regex
0.000s PASS debug64 std.signals
Ignoring std.socket(2457) test failure (likely caused by flaky environment): Socket select error: Invalid argument
0.031s PASS debug64 std.socket
uncaught exception
core.thread.threadbase.ThreadException@core/thread/osthread.d(546): Unable to join thread
----------------
conv.d:151 object.Throwable core.thread.osthread.Thread.join(bool) [0x106263ed7]
math.d:0 int std.parallelism._sharedStaticDtor_L1048_C1().__foreachbody1(ref core.thread.threadbase.ThreadBase) [0x1058bad07]
variant.d:241 int core.thread.threadbase.ThreadBase.opApply(scope int delegate(ref core.thread.threadbase.ThreadBase)) [0x1062660c3]
math.d:8286 void std.parallelism._sharedStaticDtor_L1048_C1() [0x1058bac67]
minfo.d:867 void rt.minfo.runModuleFuncsRev!(rt.minfo.ModuleGroup.runDtors().__lambda1).runModuleFuncsRev(const(immutable(object.ModuleInfo)*)[]) [0x10629c053]
minfo.d:743 void rt.minfo.ModuleGroup.runDtors() [0x10629bfdb]
sections_elf_shared.d:701 void rt.sections_elf_shared.runModuleDestructors(rt.sections_elf_shared.DSO*, bool) [0x10629faeb]
sections_elf_shared.d:550 _d_dso_registry [0x10629f82f]
test_runner.d:108 ldc.register_dso [0x104544793]
****** FAIL debug64 std.stdio
core.exception.AssertError@std/stdio.d(1610): Fork crashed
----------------
variant.d:338 onAssertErrorMsg [0x106254a9f]
variant.d:838 _d_assert_msg [0x106254eb7]
backtracking.d:0 void std.stdio.File.__unittest_L1588_C13().runForked(void delegate()) [0x105dcf00f]
backtracking.d:222 void std.stdio.File.__unittest_L1588_C13() [0x105dcedf3]
backtracking.d:0 std.stdio.__unittest [0x105e09933]
test_runner.d:73 void test_runner.doTest(object.ModuleInfo*, ref core.runtime.UnitTestResult) [0x104544503]
test_runner.d:57 int test_runner.testAll().__foreachbody1(object.ModuleInfo*) [0x10454465f]
conv.d:0 int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0x1062827a3]
minfo.d:777 int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0x10629c323]
sections_darwin_64.d:204 int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0x10629ebcb]
minfo.d:770 int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0x10629c28b]
/Users/ponce/Desktop/compilers/ldc2-1.24.0-beta1-osx-x86_64/bin/ldc-build-runtime.tmp/ldc-src/runtime/druntime/src/core/internal/array/equality.d:22 int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0x106280a1b]
test_runner.d:55 core.runtime.UnitTestResult test_runner.testAll() [0x104544453]
test_runner.d:25 core.runtime.UnitTestResult test_runner.tester() [0x104544153]
variant.d:262 runModuleUnitTests [0x10625f4af]
format.d:0 void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x1062903b3]
conv.d:150 void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x106290323]
variant.d:342 _d_run_main2 [0x106290197]
variant.d:0 _d_run_main [0x10628feb7]
test_runner.d:99 main [0x104544737]
0.023s PASS debug64 std.string
0.004s PASS debug64 std.traits
0.005s PASS debug64 std.typecons
0.000s PASS debug64 std.typetuple
0.421s PASS debug64 std.uni
0.329s PASS debug64 std.uri
0.006s PASS debug64 std.utf
0.004s PASS debug64 std.uuid
0.001s PASS debug64 std.variant
0.001s PASS debug64 std.xml
0.017s PASS debug64 std.zip
0.026s PASS debug64 std.zlib
ponce@ponces-Mac ldc-build-runtime.tmp % ./phobos2-test-runner-debug-shared 
0.004s PASS debug64 std.algorithm.comparison
0.005s PASS debug64 std.algorithm.iteration
0.095s PASS debug64 std.algorithm.mutation
0.007s PASS debug64 std.algorithm.searching
0.213s PASS debug64 std.algorithm.setops
0.442s PASS debug64 std.algorithm.sorting
0.011s PASS debug64 std.array
0.013s PASS debug64 std.ascii
0.002s PASS debug64 std.base64
0.006s PASS debug64 std.bigint
0.005s PASS debug64 std.bitmanip
0.000s PASS debug64 std.complex
0.002s PASS debug64 std.concurrency
0.001s PASS debug64 std.container.array
0.000s PASS debug64 std.container.binaryheap
0.000s PASS debug64 std.container.dlist
0.000s PASS debug64 std.container
0.015s PASS debug64 std.container.rbtree
0.000s PASS debug64 std.container.slist
0.001s PASS debug64 std.container.util
0.010s PASS debug64 std.conv
0.000s PASS debug64 std.csv
0.083s PASS debug64 std.datetime.date
0.002s PASS debug64 std.datetime.interval
0.282s PASS debug64 std.datetime
0.004s PASS debug64 std.datetime.stopwatch
1.180s PASS debug64 std.datetime.systime
0.099s PASS debug64 std.datetime.timezone
0.000s PASS debug64 std.demangle
0.008s PASS debug64 std.digest.crc
0.000s PASS debug64 std.digest.hmac
0.039s PASS debug64 std.digest.md
0.000s PASS debug64 std.digest.murmurhash
0.108s PASS debug64 std.digest
0.087s PASS debug64 std.digest.ripemd
0.462s PASS debug64 std.digest.sha
0.011s PASS debug64 std.encoding
0.004s PASS debug64 std.exception
0.001s PASS debug64 std.experimental.allocator.building_blocks.affix_allocator
0.002s PASS debug64 std.experimental.allocator.building_blocks.aligned_block_list
****** FAIL debug64 std.experimental.allocator.building_blocks.allocator_list
core.exception.AssertError@std/experimental/allocator/building_blocks/allocator_list.d(861): Assertion failure
----------------
??:? onAssertError [0x100f86b33]
??:? _d_assert [0x100f87063]
??:? void std.experimental.allocator.building_blocks.allocator_list.__unittest_L821_C9() [0x1021976ef]
??:? std.experimental.allocator.building_blocks.allocator_list.__unittest [0x1021b693f]
test_runner.d:73 void test_runner.doTest(object.ModuleInfo*, ref core.runtime.UnitTestResult) [0x100f57917]
test_runner.d:57 int test_runner.testAll().__foreachbody1(object.ModuleInfo*) [0x100f57a73]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0x100fbc217]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0x100fd6003]
??:? int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0x100fd8973]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0x100fd5f6b]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0x100fba48f]
test_runner.d:55 core.runtime.UnitTestResult test_runner.testAll() [0x100f57867]
test_runner.d:25 core.runtime.UnitTestResult test_runner.tester() [0x100f57567]
??:? runModuleUnitTests [0x100f9567f]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x100fca093]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x100fca003]
??:? _d_run_main2 [0x100fc9e77]
??:? _d_run_main [0x100fc9b97]
/Users/ponce/Desktop/compilers/ldc2-1.24.0-beta1-osx-x86_64/bin/ldc-build-runtime.tmp/ldc-src/runtime/druntime/src/core/internal/entrypoint.d:42 main [0x100f57be3]
0.251s PASS debug64 std.experimental.allocator.building_blocks.ascending_page_allocator
0.009s PASS debug64 std.experimental.allocator.building_blocks.bitmapped_block
0.000s PASS debug64 std.experimental.allocator.building_blocks.bucketizer
0.004s PASS debug64 std.experimental.allocator.building_blocks.fallback_allocator
0.000s PASS debug64 std.experimental.allocator.building_blocks.free_list
0.000s PASS debug64 std.experimental.allocator.building_blocks.free_tree
0.002s PASS debug64 std.experimental.allocator.building_blocks.kernighan_ritchie
0.000s PASS debug64 std.experimental.allocator.building_blocks.null_allocator
0.000s PASS debug64 std.experimental.allocator.building_blocks.quantizer
0.017s PASS debug64 std.experimental.allocator.building_blocks.region
0.000s PASS debug64 std.experimental.allocator.building_blocks.scoped_allocator
0.003s PASS debug64 std.experimental.allocator.building_blocks.segregator
0.001s PASS debug64 std.experimental.allocator.building_blocks.stats_collector
0.000s PASS debug64 std.experimental.allocator.common
0.000s PASS debug64 std.experimental.allocator.gc_allocator
0.000s PASS debug64 std.experimental.allocator.mallocator
0.000s PASS debug64 std.experimental.allocator.mmap_allocator
0.027s PASS debug64 std.experimental.allocator
0.000s PASS debug64 std.experimental.allocator.showcase
0.000s PASS debug64 std.experimental.allocator.typed
Overflow on binary operator: int(-2147483648) / const(int)(-1)
Overflow on binary operator: int(-2147483648) * const(int)(-1)
Overflow on binary operator: int(-2147483648) / const(int)(-1)
0.000s PASS debug64 std.experimental.checkedint
0.047s PASS debug64 std.experimental.logger.core
0.001s PASS debug64 std.experimental.logger.filelogger
0.000s PASS debug64 std.experimental.logger.multilogger
0.000s PASS debug64 std.experimental.logger.nulllogger
0.000s PASS debug64 std.experimental.typecons
0.020s PASS debug64 std.file
0.006s PASS debug64 std.format
0.004s PASS debug64 std.functional
0.001s PASS debug64 std.getopt
0.000s PASS debug64 std.internal.cstring
0.000s PASS debug64 std.internal.math.biguintcore
0.000s PASS debug64 std.internal.math.biguintnoasm
0.000s PASS debug64 std.internal.math.errorfunction
0.000s PASS debug64 std.internal.math.gammafunction
0.000s PASS debug64 std.internal.scopebuffer
0.000s PASS debug64 std.internal.test.dummyrange
0.000s PASS debug64 std.internal.test.range
0.000s PASS debug64 std.json
0.004s PASS debug64 std.math
0.000s PASS debug64 std.mathspecial
0.004s PASS debug64 std.meta
0.000s PASS debug64 std.mmfile
0.040s PASS debug64 std.net.curl
0.004s PASS debug64 std.net.isemail
0.001s PASS debug64 std.numeric
0.000s PASS debug64 std.outbuffer
0.000s PASS debug64 std
0.848s PASS debug64 std.parallelism
0.063s PASS debug64 std.path
stderr test, please ignore
5.213s PASS debug64 std.process
1.601s PASS debug64 std.random
0.000s PASS debug64 std.range.interfaces
0.012s PASS debug64 std.range
0.004s PASS debug64 std.range.primitives
0.018s PASS debug64 std.regex.internal.generator
0.000s PASS debug64 std.regex.internal.ir
0.001s PASS debug64 std.regex.internal.kickstart
0.004s PASS debug64 std.regex.internal.parser
0.268s PASS debug64 std.regex.internal.tests
0.037s PASS debug64 std.regex.internal.tests2
0.007s PASS debug64 std.regex
0.000s PASS debug64 std.signals
Ignoring std.socket(2457) test failure (likely caused by flaky environment): Socket select error: Invalid argument
0.030s PASS debug64 std.socket
uncaught exception
core.thread.threadbase.ThreadException@core/thread/osthread.d(546): Unable to join thread
----------------
??:? object.Throwable core.thread.osthread.Thread.join(bool) [0x100f9cd9b]
??:? int std.parallelism._sharedStaticDtor_L1048_C1().__foreachbody1(ref core.thread.threadbase.ThreadBase) [0x1026ecaeb]
??:? int core.thread.threadbase.ThreadBase.opApply(scope int delegate(ref core.thread.threadbase.ThreadBase)) [0x100f9ef87]
??:? void std.parallelism._sharedStaticDtor_L1048_C1() [0x1026eca4b]
??:? void rt.minfo.runModuleFuncsRev!(rt.minfo.ModuleGroup.runDtors().__lambda1).runModuleFuncsRev(const(immutable(object.ModuleInfo)*)[]) [0x100fd5d33]
??:? void rt.minfo.ModuleGroup.runDtors() [0x100fd5cbb]
??:? void rt.sections_elf_shared.runModuleDestructors(rt.sections_elf_shared.DSO*, bool) [0x100fda32f]
??:? _d_dso_registry [0x100fd9d2f]
??:? ldc.register_dso [0x10138cd17]
****** FAIL debug64 std.stdio
core.exception.AssertError@std/stdio.d(1610): Fork crashed
----------------
??:? onAssertErrorMsg [0x100f86c0f]
??:? _d_assert_msg [0x100f87027]
??:? void std.stdio.File.__unittest_L1588_C13().runForked(void delegate()) [0x102c00df3]
??:? void std.stdio.File.__unittest_L1588_C13() [0x102c00bd7]
??:? std.stdio.__unittest [0x102c3b717]
test_runner.d:73 void test_runner.doTest(object.ModuleInfo*, ref core.runtime.UnitTestResult) [0x100f57917]
test_runner.d:57 int test_runner.testAll().__foreachbody1(object.ModuleInfo*) [0x100f57a73]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0x100fbc217]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0x100fd6003]
??:? int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0x100fd8973]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0x100fd5f6b]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0x100fba48f]
test_runner.d:55 core.runtime.UnitTestResult test_runner.testAll() [0x100f57867]
test_runner.d:25 core.runtime.UnitTestResult test_runner.tester() [0x100f57567]
??:? runModuleUnitTests [0x100f9567f]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x100fca093]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x100fca003]
??:? _d_run_main2 [0x100fc9e77]
??:? _d_run_main [0x100fc9b97]
/Users/ponce/Desktop/compilers/ldc2-1.24.0-beta1-osx-x86_64/bin/ldc-build-runtime.tmp/ldc-src/runtime/druntime/src/core/internal/entrypoint.d:42 main [0x100f57be3]
0.022s PASS debug64 std.string
0.003s PASS debug64 std.traits
0.005s PASS debug64 std.typecons
0.000s PASS debug64 std.typetuple
0.422s PASS debug64 std.uni
0.328s PASS debug64 std.uri
0.006s PASS debug64 std.utf
0.004s PASS debug64 std.uuid
0.001s PASS debug64 std.variant
0.001s PASS debug64 std.xml
0.016s PASS debug64 std.zip
0.026s PASS debug64 std.zlib

In particular that leads to :

core.exception.AssertError@std/experimental/allocator/building_blocks/allocator_list.d(861): Assertion failure => https://github.com/ldc-developers/phobos/blob/ldc/std/experimental/allocator/building_blocks/allocator_list.d#L861

Seems like a hardcoded page issue.

core.thread.threadbase.ThreadException@core/thread/osthread.d(546): Unable to join thread
core.exception.AssertError@std/stdio.d(1610): Fork crashed

=> https://github.com/ldc-developers/phobos/blob/ldc/std/stdio.d#L1610

Not sure if one is allowed to launch threads under Big Sur, I think there are new requirements.

Oct 07 '20 22:10 p0nce

Damn, the x86_84 target still needs the $INODE64 compatibility names, we need to static if (AArch64)

Oct 07 '20 22:10 p0nce

Seems like a hardcoded page issue.

I thought I did take care of that for iOS [1]. Hmm, I guess it should say Darwin now. That's the only occurrence of iOSDerived in druntime, but there are a few in Phobos. You should double check these to see if they're correct or should be changed to Darwin.

[1] https://github.com/dlang/druntime/blob/c304e21a949e0a1b23908503ed825a050f1e8bdb/src/core/memory.d#L183-L184

Oct 08 '20 06:10 jacob-carlborg

@jacob-carlborg Sounds like it. Page size is also 16kb on macOS Big Sur (EDIT: not sure how it affects the x86 unittest yet!)

Oct 08 '20 08:10 p0nce

not sure how it affects the x86 unittest yet!

I don't see why it would. It's already passing on iOS and macOS x86-64. The minimumPageSize symbol is already different on ARM and x86.

Oct 08 '20 09:10 jacob-carlborg

So it's possible to have a page size of 4kb on emulated x86 and 16kb on ARM? The problem with this machine is it can run both. So I need to test both in general.

Oct 08 '20 09:10 p0nce

So it's possible to have a page size of 4kb on emulated x86 and 16kb on ARM?

Yes, why wouldn't it be? The code is already guarded with the appropriate version conditions, it just needs a minor adjustment.

Perhaps I missing something. I don't see the problem.

Oct 08 '20 10:10 jacob-carlborg

You are probably right, I was just imagining a case where on Big Sur page size would be 16kb even in x86_64/Rosetta2; I just need to check this right now.

Oct 08 '20 11:10 p0nce

You are probably right, I was just imagining a case where on Big Sur page size would be 16kb even in x86_64/Rosetta2; I just need to check this right now.

Aha, you were worried about running under Rosetta2. I see now. Apple mentioned in a WWDC video that the page size would match the real hardware. But you can double check.

Oct 08 '20 11:10 jacob-carlborg