ShellOut
ShellOut copied to clipboard
Close pipes after reading
Description
I have a Swift CLI that utilizes ShellOut to interact with binary executables. When I use shellOut
too many times, I sometimes hit an exception that looks something like:
*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Failed to set posix_spawn_file_actions for fd -1 at index 0 with errno 9'
*** First throw call stack:
(
0 CoreFoundation 0x00000001859672ec __exceptionPreprocess + 176
1 libobjc.A.dylib 0x000000018544e788 objc_exception_throw + 60
2 Foundation 0x0000000186a92eb4 -[NSConcreteTask launchWithDictionary:error:] + 4240
I've been able to determine this is due to the Pipe
objects from ShellOut not being closed properly.
I've included an example project that demonstrates the issue: NSExceptionExample.zip
Steps:
- Unzip the example library and
cd NSExceptionExample
- Run
swift run NSExceptionExample
- While that command is still running, in a separate terminal window, run
ps aux | grep NSExceptionExample
and take note of the PID. - Run
lsof -p <pid> | wc -l
- Notice how this number is quite large. Also notice that running again shows the number increasing.
- If the
swift
command runs long enough, you'll hit an exception like the above.
If you change the example to use this fork and branch, you'll find that this file descriptor increase does not occur when repeating the above steps.
Note: I would love to add a unit test to ensure many commands can be run in the same process by shellOut
, but I haven't found a good way to do so without creating an extremely long test or creating a very convoluted test harness. Let me know if you have any suggestions for a unit test.
System Info
MacBook Pro M1 macOS 14.5 Xcode 15.3, Swift 5.10
$ ulimit -a
-t: cpu time (seconds) unlimited
-f: file size (blocks) unlimited
-d: data seg size (kbytes) unlimited
-s: stack size (kbytes) 8176
-c: core file size (blocks) 0
-v: address space (kbytes) unlimited
-l: locked-in-memory size (kbytes) unlimited
-u: processes 5333
-n: file descriptors 256
$ sysctl -A | grep kern.maxfiles
kern.maxfiles: 245760
kern.maxfilesperproc: 122880