v1.0.15 produce error /usr/lib/libSystem.B.dylib, v1.0.14 worked fine

[eugene-piatenko]

What version of Bun is running?

1.0.15+b3bdf22eb

What platform is your computer?

Darwin 19.6.0 x86_64 i386

  System:
    OS: macOS 10.15.7
    CPU: (8) x64 Intel(R) Core(TM) i7-3720QM CPU @ 2.60GHz
    Memory: 398.05 MB / 16.00 GB
    Shell: 5.7.1 - /bin/zsh

What steps can reproduce the bug?

I have a small project. It runs fine on v1.0.14, but when I tried to run on v1.0.15 it shows error below

What is the expected behavior?

normal program run

What do you see instead?

dyld: lazy symbol binding failed: Symbol not found: _pwritev$NOCANCEL
  Referenced from: /Users/jj/Downloads/bun-darwin-x64-baseline/./bun
  Expected in: /usr/lib/libSystem.B.dylib

Additional information

> objdump -t /usr/lib/libSystem.B.dylib | grep write
0000000000001a78 g     F __TEXT,__text R8289209$_write
0000000000000000         *UND* _write

[dimaskiddo]

Got Same issues on Mac OS 10.15 with Latest Bun Version 1.0.16

What version of Bun is running?

1.0.16 (Baseline)

What steps can reproduce the bug?

Upgrade bun from version 1.0.14 to latest 1.0.16 When run the typescript code got an dyld symbol error, but working correctly in version 1.0.14

What is the expected behavior?

Bun can run the typescript code and hoping bun not drop support for old MacOS

What do you see instead?

$ bun run pkg/app.ts

dyld: lazy symbol binding failed: Symbol not found: _pwritev$NOCANCEL
  Referenced from: /usr/local/bin/bun
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _pwritev$NOCANCEL
  Referenced from: /usr/local/bin/bun
  Expected in: /usr/lib/libSystem.B.dylib

[alanyang]

me too, bun 1.0.20, macos 10.15.7

$ bun run pkg/app.ts

dyld: lazy symbol binding failed: Symbol not found: _pwritev$NOCANCEL
  Referenced from: /usr/local/bin/bun
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _pwritev$NOCANCEL
  Referenced from: /usr/local/bin/bun
  Expected in: /usr/lib/libSystem.B.dylib

[adhanji8]

This issue appears to persist on the latest canary, 1.0.27, with MacOS 10.15.7 (MacOS Catalina).

[kzc]

given:

bun 1.0.30 canary Mac OS X 10.14.6 x86_64

$ cat test-pwritev.js
var assert = require("assert");
var fs = require("fs");
var FNAME = "zzz.tmp";
var B = s => Buffer.from(s, "utf8");
var fd = fs.openSync(FNAME, "w");
fs.writeSync(fd, "the quick brown fox");
fs.writevSync(fd, [ B("GREEN") ], 10);
fs.writeSync(fd, " jumps");        // verify write pos preserved
fs.writevSync(fd, [ B("TH"), B("E") ], 0);
fs.writeSync(fd, " triumphantly"); // verify write pos preserved
var contents = fs.readFileSync(FNAME).toString("utf8");
console.log(contents);
assert.equal(contents, "THE quick GREEN fox jumps triumphantly");
fs.closeSync(fd);
fs.unlinkSync(FNAME);

expected:

$ node test-pwritev.js
THE quick GREEN fox jumps triumphantly

workaround:

$ cat pwritev.c
#include <sys/uio.h>
#include <unistd.h>
ssize_t pwritev$NOCANCEL(int fd, const struct iovec *iov, int iovcnt, off_t offset) {
    ssize_t written;
    off_t origPos = lseek(fd, 0, SEEK_CUR);
    if (origPos >= 0
        && lseek(fd, offset, SEEK_SET) >= 0
        && (written = writev(fd, iov, iovcnt)) >= 0
        && lseek(fd, origPos, SEEK_SET) >= 0) {
            return written;
    }
    return -1;
}

$ cc -dynamiclib -O pwritev.c -o pwritev.dylib

$ export DYLD_INSERT_LIBRARIES=`pwd`/pwritev.dylib DYLD_FORCE_FLAT_NAMESPACE=1

$ bun test-pwritev.js
THE quick GREEN fox jumps triumphantly

[kzc]

This is interesting. The pwritev$NOCANCEL polyfill works fine for single process bun use, but it doesn't work on macOS 10.x for bun repl and other exec'd programs:

$ export DYLD_INSERT_LIBRARIES=`pwd`/pwritev.dylib DYLD_FORCE_FLAT_NAMESPACE=1

$ bun test-pwritev.js
THE quick GREEN fox jumps triumphantly

$ bun repl
dyld: lazy symbol binding failed: Symbol not found: _pwritev$NOCANCEL
  Referenced from: /usr/local/bin/bun (which was built for Mac OS X 11.0)
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _pwritev$NOCANCEL
  Referenced from: /usr/local/bin/bun (which was built for Mac OS X 11.0)
  Expected in: /usr/lib/libSystem.B.dylib

Killed: 9

Apparently dylib polyfills won't work on many macOS applications by default as a security measure if they do not have the com.apple.security.cs.allow-dyld-environment-variables entitlement:

• https://hynek.me/articles/macos-dyld-env/
• https://developer.apple.com/forums/thread/714782
• https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_allow-dyld-environment-variables

Although both bun and node do have this entitlement and allow propagation of DYLD_ prefixed environment variables:

$ FOO=node DYLD_FORCE_FLAT_NAMESPACE=12345 node --print '[process.env.FOO, process.env.DYLD_FORCE_FLAT_NAMESPACE]'
[ 'node', '12345' ]

$ FOO=bun DYLD_FORCE_FLAT_NAMESPACE=12345 bun --print '[process.env.FOO, process.env.DYLD_FORCE_FLAT_NAMESPACE]'
[ "bun", "12345" ]

you can see that DYLD_ prefixed variables are not propagated to exec'd child processes that use /bin/sh directly or indirectly:

$ FOO=/bin/sh DYLD_FORCE_FLAT_NAMESPACE=12345 /bin/sh -c 'echo [$FOO, $DYLD_FORCE_FLAT_NAMESPACE]'
[/bin/sh, ]

$ FOO=exec DYLD_FORCE_FLAT_NAMESPACE=12345 bun -e 'child_process.exec("bun --print [process.env.FOO,process.env.DYLD_FORCE_FLAT_NAMESPACE]",(e,o)=>console.log(o))'
[ "exec", undefined ]

but DYLD_ env vars are correctly propagated with child_process.spawn because /bin/sh is not an intermediary:

$ FOO=spawn DYLD_FORCE_FLAT_NAMESPACE=12345 bun -e 'child_process.spawn("bun",["--print","[process.env.FOO,process.env.DYLD_FORCE_FLAT_NAMESPACE]"]).stdout.on("data",d=>console.log(d+""))'
[ "spawn", "12345" ]

So the pwritev.dylib workaround will not work with /bin/sh exec'd processes. The pwritev.c polyfill would have to be incorporated into the bun binary like the other ones to be more reliable.

[kzc]

If you don't wish to alter /bin/sh and /usr/bin/env on macos 10.x, the DYLD_* environment variable exporting problem can be worked around by interposing execve to change the path argument if it happens to be /bin/sh to a compatible shell with the appropriate entitlements. It would also have to handle script headers of the form #!/usr/bin/env interpreter (as is used by node_modules/.bin/bun-repl) to invoke the interpreter binary directly without an intermediate shell. All posix_spawn* functions would also have to be replaced with an alternate implementation such as musl because macos posix_spawn is implemented as a syscall and as such avoids calling execve in user space. This has the beneficial side effect of correctly implementing posix_spawn_file_actions_addchdir_np and posix_spawn_file_actions_addfchdir_np to avoid errors installing node modules with bun.