swc-project
`npx swc "<dir>" -d '/tmp/output'` gives 4618 `EMFILE: too many open files` errors
Describe the bug
EMFILE: too many open files, open '/tmp/output/@firebase/auth/dist/esm2017/src/core/auth/register.d.js'
EMFILE: too many open files, open '/tmp/output/@firebase/auth/dist/esm2017/src/core/auth/emulator.d.js'
EMFILE: too many open files, open '/tmp/output/@firebase/auth/dist/esm2017/src/api/authentication/custom_token.d.js'
[…]
Outputting them all to a logfile and I get grep -F 'EMFILE' /tmp/a.log | wc -l with 4618.
Input code
$ npx swc "<dir>" -d '/tmp/output'
On a reasonably large codebase: find -type f -name '*.ts' | wc -l gives 9029.
Config
N/A (or default)
Playground link
No response
Expected behavior
It should automatically determine that my filesystem and/or disk and/or RAM and/or CPU is incapable of this level of parallelism.
Actual behavior
Lots of non TypeScript related errors. Note in this codebase I am converting everything from JavaScript to TypeScript so am expected hundreds of errors in total (possibly even thousands). tsc [4.8.4] only reports 993 errors.
Version
@swc/cli: 0.1.57 @swc/core: 1.3.14
Additional context
macOS 13.0 (22A380) with Node.js v18.12.1 on an APFS filesystem over PCI-Express (MacBook Pro laptop) and LPDDR3 RAM on a Quad-Core Intel Core i5.
I had the same problem, and I worked around it by batching compilation by folder.
#!/bin/bash
set -exuo pipefail
FOLDERS=$(exa -D my-package | rg -g '!node_module' '.*')
FILES=$(exa my-package | rg '(\.json$)|(\.ts$)|(\.md)|(\.lock)')
# Compile the files at the top level. Don't recurse.
cd my-package
npx swc \
--source-maps=true \
--copy-files \
--config-file="../.swcrc" \
--out-dir="../dist/nodejs" \
$FILES
cd ../
for FOLDER in $FOLDERS
do
npx swc \
--source-maps=true \
--copy-files \
--config-file=".swcrc" \
--out-dir="dist/" \
"./my-package/${FOLDER}"
done
seems that another workaround for this issue is to run swc with the --sync flag on the cli
