zx
zx copied to clipboard
Improve README documentation for TypeScript usage
Expected Behavior
The 'TypeScript' section of the README file would provide accurate, complete, and up to date information of how to use zx with TypeScript.
Actual Behavior
The current 'how to use TypeScript' documentation appears inadequate:
https://github.com/google/zx/blob/854e4eca9e42f056fa9d1de9f5e7abd1e7461cbd/README.md?plain=1#L454-L468
Crawling through the release history it seems like support for TypeScript via the zx binary was dropped in v5.0.0:
- https://github.com/google/zx/releases/tag/5.0.0
This release drops build of CommonJS version and support for
.tsextension byzxbin.TypeScript is still supported, for example, via ts-node:
node --loader ts-node/esm script.tsAlso, a new Node version requirement is >= 16.0.0.
Looking at the README at this commit seemed to have better usage information for how to use this with ts-node:
https://github.com/google/zx/blob/7977cb5adb9545082b8edb8bfe647dedfcb98b42/README.md?plain=1#L398-L421
Which seems to have been removed ~17 days ago in this commit: https://github.com/google/zx/commit/db0e65163d31e37cc6d71ae2e20e2ada4186efa6#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L542-R458
Following the current README as written, I tried making a foo.mjs test file:
#!/usr/bin/env zx
import 'zx/globals'
void async function () {
const foo: string = "foo";
await $`ls -la`
}()
Running this with ./foo.mjs (zx bin in the shebang) results in the following error, presumably because .mjs doesn't specify it as a TypeScript file:
SyntaxError: Missing initializer in const declaration
at ESMLoader.moduleStrategy (node:internal/modules/esm/translators:117:18)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:337:14)
at async link (node:internal/modules/esm/module_job:70:21)
I also tried renaming it to foo.ts, which resulted in this error:
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /Users/devalias/dev/0xdevalias/minimal-zx-ts/foo.ts
at new NodeError (node:internal/errors:372:5)
at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:76:11)
at defaultGetFormat (node:internal/modules/esm/get_format:118:38)
at defaultLoad (node:internal/modules/esm/load:21:20)
at ESMLoader.load (node:internal/modules/esm/loader:407:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:326:22)
at new ModuleJob (node:internal/modules/esm/module_job:66:26)
at ESMLoader.#createModuleJob (node:internal/modules/esm/loader:345:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:304:34)
at async Promise.all (index 0) {
code: 'ERR_UNKNOWN_FILE_EXTENSION'
}
Looking at the TypeScript docs for 'ECMAScript Modules in Node.js' it talks about various other file extensions that can be used:
- https://www.typescriptlang.org/docs/handbook/esm-node.html#new-file-extensions
The type field in package.json is nice because it allows us to continue using the
.tsand.jsfile extensions which can be convenient; however, you will occasionally need to write a file that differs from what type specifies. You might also just prefer to always be explicit.Node.js supports two extensions to help with this:
.mjsand.cjs..mjsfiles are always ES modules, and.cjsfiles are always CommonJS modules, and there’s no way to override these.In turn, TypeScript supports two new source file extensions:
.mtsand.cts. When TypeScript emits these to JavaScript files, it will emit them to .mjsand.cjsrespectively.
Based on this, I figured that maybe the .mts extension would work, as it should be the 'TypeScript flavoured' version of the recommended .mjs; but renaming to foo.mts and running it again resulted in this error:
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".mts" for /Users/devalias/dev/0xdevalias/minimal-zx-ts/foo.mts
at new NodeError (node:internal/errors:372:5)
at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:76:11)
at defaultGetFormat (node:internal/modules/esm/get_format:118:38)
at defaultLoad (node:internal/modules/esm/load:21:20)
at ESMLoader.load (node:internal/modules/esm/loader:407:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:326:22)
at new ModuleJob (node:internal/modules/esm/module_job:66:26)
at ESMLoader.#createModuleJob (node:internal/modules/esm/loader:345:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:304:34)
at async Promise.all (index 0) {
code: 'ERR_UNKNOWN_FILE_EXTENSION'
}
Based on the old README example of using ts-node I decided to try that method, and followed the latest instructions/details from:
- https://github.com/TypeStrong/ts-node#native-ecmascript-modules
- https://github.com/TypeStrong/ts-node/issues/1007#issue-598417180
- https://github.com/TypeStrong/ts-node/issues/1007#issuecomment-1139917958
- https://github.com/TypeStrong/ts-node/issues/1007#issuecomment-1163471306
I can seemingly run the script with ts-node-esm directly as:
npm exec ts-node-esm -- ./foo.mts
And by modifying the shebang to call ts-node-esm:
#!/usr/bin/env npx --package=ts-node -- ts-node-esm
import 'zx/globals'
void async function () {
const foo: string = "foo";
await $`ls -la`
}()
We can run the file with a .ts extension (when "type": "module", is set in package.json).
If we try to use a .js or .mjs extension we get the same error as earlier (as we would expect):
SyntaxError: Missing initializer in const declaration
at ESMLoader.moduleStrategy (node:internal/modules/esm/translators:117:18)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:337:14)
at async link (node:internal/modules/esm/module_job:70:21)
If we use the .mts extension however, we can remove "type": "module", from package.json and it will still work (as .mts are files are always ES modules)
We can make this run even faster if we use ts-node's swc suport:
- https://typestrong.org/ts-node/docs/transpilers/
-
ts-nodesupports third-party transpilers as plugins. Transpilers such asswccan transform TypeScript into JavaScript much faster than the TypeScript compiler. You will still benefit fromts-node's automatictsconfig.jsondiscovery, sourcemap support, and globalts-nodeCLI.
-
- https://typestrong.org/ts-node/docs/swc
-
SWC support is built-in via the
--swcflag or"swc": truetsconfigoption. -
SWC is a TypeScript-compatible transpiler implemented in Rust. This makes it an order of magnitude faster than vanilla
transpileOnly. -
To use it, first install
@swc/coreor@swc/wasm. If usingimportHelpers, also install@swc/helpers. If target is less than"es2015"and usingasync/awaitor generator functions, also installregenerator-runtime.
-
- https://github.com/swc-project/swc
- https://swc.rs/
Based on this we could install the @swc/core lib, and modify our script's shebang to add the --swc arg as follows:
npm i --save-dev @swc/core
#!/usr/bin/env npx --package=ts-node -- ts-node-esm --swc
import 'zx/globals'
void async function () {
const foo: string = "foo";
await $`ls -la`
}()
Super basic testing on my laptop with time shows the following speed improvements when using --swc with ts-node:
// time ./foo.mts
// Without --swc
./foo.mts 3.45s user 0.55s system 126% cpu 3.151 total
./foo.mts 3.29s user 0.34s system 153% cpu 2.374 total
./foo.mts 3.16s user 0.34s system 156% cpu 2.243 total
// With --swc
./foo.mts 1.14s user 0.46s system 69% cpu 2.299 total
./foo.mts 1.14s user 0.27s system 103% cpu 1.358 total
./foo.mts 1.13s user 0.27s system 103% cpu 1.356 total
Contrasting this against a plain .mjs file using the zx bin directly, we can see that the .mjs is still faster, but for the dev efficiency improvements of using TypeScript, I feel like the tradeoff isn't too bad when using tas-node with --swc:
#!/usr/bin/env zx
import 'zx/globals'
const foo = "foo";
await $`ls -la`
console.log(foo);
./foo.mjs 0.46s user 0.25s system 72% cpu 0.977 total
./foo.mjs 0.45s user 0.11s system 118% cpu 0.481 total
./foo.mjs 0.44s user 0.11s system 119% cpu 0.461 total
Trying to remove the wrapping async function results in the following error:
/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:843
return new TSError(diagnosticText, diagnosticCodes, diagnostics);
^
TSError: ⨯ Unable to compile TypeScript:
foo.mts:7:3 - error TS1378: Top-level 'await' expressions are only allowed when the 'module' option is set to 'es2022', 'esnext', 'system', 'node16', or 'nodenext', and the 'target' option is set to 'es2017' or higher.
7 await $`ls -la`
~~~~~
at createTSError (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:843:12)
at reportTSError (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:847:19)
at getOutput (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:1057:36)
at Object.compile (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:1411:41)
at transformSource (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/esm.ts:400:37)
at /Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/esm.ts:278:53
at async addShortCircuitFlag (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/esm.ts:409:15)
at async ESMLoader.load (node:internal/modules/esm/loader:407:20)
at async ESMLoader.moduleProvider (node:internal/modules/esm/loader:326:11)
at async link (node:internal/modules/esm/module_job:70:21) {
diagnosticCodes: [ 1378 ]
}
Which we can fix by setting our tsconfig.json to something like this:
{
"compilerOptions": {
"target": "ES2021",
"module": "node16",
},
}
Which then allows us to simplify the basic code to:
#!/usr/bin/env npx --package=ts-node -- ts-node-esm
import 'zx/globals'
const foo: string = "foo";
await $`ls -la`
We could also use something like the following (from https://github.com/tsconfig/bases#node-16--esm--strictest-tsconfigjson) rather than handcrafting our own compatible tsconfig.json:
npm i @tsconfig/node16-strictest-esm
// tsconfig.json
{
"extends": "@tsconfig/node16-strictest-esm/tsconfig.json"
}
An alternative to using ts-node could be to use tsm (which wraps esbuild), but I didn't look too deeply into how to make it work:
- https://github.com/lukeed/tsm
- https://github.com/lukeed/tsm/blob/master/docs/usage.md#shell--shebang
- https://github.com/evanw/esbuild
Steps to Reproduce the Problem
- Be a newbie to using
zx - Wonder how to use it effectively with TypeScript
- Get confused
Specifications
- Version: 7.0.5
- Platform: node v16.15.1, macOS 12.3.1
Other older TypeScript related issues that I looked through first
- https://github.com/google/zx/pull/409
- https://github.com/google/zx/issues/273
- https://github.com/google/zx/issues/206
- https://github.com/google/zx/issues/205
- https://github.com/google/zx/issues/197
- https://github.com/google/zx/issues/194
- https://github.com/google/zx/issues/152
- https://github.com/google/zx/issues/125
- https://github.com/google/zx/pull/114
- https://github.com/google/zx/issues/110
- https://github.com/google/zx/issues/75
Also potentially relevant:
I haven't gone far with this yet, but this is the shebang I'm using in a script to get
ts-nodeto use ESM mode:#!/usr/bin/env -S npx ts-node --esm --compilerOptions={"module":"ESNext","target":"ESNext","moduleResolution":"node"} import { $ } from 'zx'; await $`echo ${'stuff'}`;Originally posted by @tvsbrent in https://github.com/google/zx/issues/125#issuecomment-1169890068
Awesome! Lets add condensed end version of this to the readme.
I tried tsup, and it works for me. my tsup.config.json like this:
{
"entry": ["src/index.ts"],
"splitting": true,
"outDir": "bin",
"sourcemap": true,
"format": "esm",
"target": "node16",
"clean": true
}
packgae.json
"scripts": {
"dev": "tsup --watch"
}
I almost managed to marry zx with ts-node like this:
- install ts-node and zx globally
- create
foo.mts(module ts file) with content:
#!/usr/bin/env -S ts-node-esm --compilerOptions='{"module": "es2022"}'
import 'zx/globals';
await $`echo it works ${JSON.stringify(foo(1))}`;
function foo(num: number): number[] {
return [num, num * 2, num * 3];
}
- execute "./foo.mts"
But the problem I'm facing is that by default, Node doesn't resolve import 'zx/globals' to /usr/lib/node_modules/.
Typescript fails with Error TS2592: Cannot find name '$'.
If I install zx in ./node_modules, it works. But I want to avoid this.
I think currently zx + ts wotks only for locally installed dependencies. :₽
But will be cool if zx also supports global ts scripts.
Also potentially relevant:
I haven't gone far with this yet, but this is the shebang I'm using in a script to get
ts-nodeto use ESM mode:#!/usr/bin/env -S npx ts-node --esm --compilerOptions={"module":"ESNext","target":"ESNext","moduleResolution":"node"} import { $ } from 'zx'; await $`echo ${'stuff'}`;Originally posted by @tvsbrent in #125 (comment)
An update to what was referenced above. I have used the following shebang for a few different scripts now, and it seems to work pretty well:
#!/usr/bin/env -S npx ts-node --esm --project=./tsconfig.build.json
In this case, my tsconfig.build.json has the following:
{
"extends": "./tsconfig.json",
"exclude": [
"node_modules",
"**/*.test.ts",
"**/*.spec.ts"
],
}
In the base tsconfig.json, I have the following properties:
"target": "ESNext",
"module": "ESNext",
"moduleResolution": "node",
In addition, in the package.json, I have set "type": "module".
Just published a quick helper script to make this easier. See repo
Install it npm install -g tzx
Use it tzx <script.ts>
Hey guys, I've got a nicely working solution for now. Requirements:
- package.json file that has
"type": "module" npm install -g zx tsup
Create your script.ts file in the same directory as your package.json, example
#!/usr/bin/env zx
import { $ } from "zx";
await $`echo "test"`;
Now this is for linux only but you can adapt it to other platforms. Inside your .bashrc add the following function:
zxe() {
INPUT=$1
OUTPUT=$(sed 's/.ts/.js/' <<< "$INPUT")
tsup-node -d . --format esm --target {{NODE_VERSION}} "$INPUT"
sed '/^import.*zx"/d' -i "$OUTPUT"
zx "$OUTPUT"
rm "$OUTPUT"
}
Replace {{NODE_VERSION}} with node18 or node16 or whatever.
Restart your terminal or you can run source ~/.bashrc.
Now you can run your scripts with zxe script.ts
all other problems problems like, intelisense and imports work with this settings https://github.com/felipeplets/esm-examples
I've found success using the .mts extension and adding the following shebang:
#!/usr/bin/env -S npx tsx
import 'zx/globals';
await $`ls -la`;
console.log(chalk.green('Hello, World!'));
Run like so:
chmod +x ./myfile.mts
./myfile.mts
Just published a quick helper script to make this easier. See repo
Install it
npm install -g tzxUse ittzx <script.ts>Hey guys, I've got a nicely working solution for now. Requirements:
- package.json file that has
"type": "module"npm install -g zx tsupCreate your script.ts file in the same directory as your package.json, example
#!/usr/bin/env zx import { $ } from "zx"; await $`echo "test"`;Now this is for linux only but you can adapt it to other platforms. Inside your
.bashrcadd the following function:zxe() { INPUT=$1 OUTPUT=$(sed 's/.ts/.js/' <<< "$INPUT") tsup-node -d . --format esm --target {{NODE_VERSION}} "$INPUT" sed '/^import.*zx"/d' -i "$OUTPUT" zx "$OUTPUT" rm "$OUTPUT" }Replace {{NODE_VERSION}} with
node18ornode16or whatever.Restart your terminal or you can run
source ~/.bashrc.Now you can run your scripts with
zxe script.ts
Your lib worked great for me, thanks!
I've tried most of these suggestions, but I keep getting stumped by $ not being recognized
e.g.
#!/usr/bin/env npx --package=ts-node -- ts-node-esm
import "zx/globals";
await $`echo ${"stuff"}`;
Results in
error TS2592: Cannot find name '$'. Do you need to install type definitions for jQuery? Try `npm i --save-dev @types/jquery`
I do have zx globally installed.
Why not use tsx instead? It's based on esbuild instead of typescript, can compile ts but doesn't check types, is fast and great for scripts. In addition, it also supports esm/cjs, replacing esmo/esno. ref: https://github.com/esbuild-kit/tsx
Why not use tsx instead? It's based on esbuild instead of typescript, can compile ts but doesn't check types, is fast and great for scripts. In addition, it also supports esm/cjs, replacing esmo/esno. ref: https://github.com/esbuild-kit/tsx
It's the $ feature that zx provides that makes it so powerful. The ability to write env vars back to the shell is something that is quite non-trivial without a tool like zx, if tsx support that too then I wouldn't mind switching but I doubt it's built for that use case.
Why not use tsx instead? It's based on esbuild instead of typescript, can compile ts but doesn't check types, is fast and great for scripts. In addition, it also supports esm/cjs, replacing esmo/esno. ref: https://github.com/esbuild-kit/tsx
It's the
$feature thatzxprovides that makes it so powerful. The ability to write env vars back to the shell is something that is quite non-trivial without a tool likezx, iftsxsupport that too then I wouldn't mind switching but I doubt it's built for that use case.
you just need
import 'zx/globals';
Here's my setup with tsx...
Create a bin directory & create a node_modules there for the script environment...
mkdir ~/bin
cd ~/bin
npm install zx
cd ..
Create a small test script...
cat > ~/bin/foo.ts
#!/usr/bin/env tsx
import 'zx/globals';
$`echo hello`;
<ctl-d>
Now to test the test script...
~/bin/foo.ts
This outputs:
$ echo hello
hello
Maybe something like this can be added to the README?
I recently encountered the same TypeError [ERR_UNKNOWN_FILE_EXTENSION] error. To solve this I added shebang and run the file with ts-node.
Here's the example of how I did this:
run.zx.ts
#!/usr/bin/env zx
import { $ } from 'zx/core'
await $`echo "hi"`
package.json
{
"scripts": {
"zx": "ts-node run.zx.ts"
}
}
With this setup I was able to execute the script using zx without any errors.
How about executing with deno instead of node?
Just to share my working configuration. My requirements are:
- *.ts files can be executed directly
- I can run test cases with jest
- The Node.js version is 18
This is what I have in package.json (only related parts are shown):
{
"type": "module",
"engines": {
"node": ">=18.0.0"
},
"scripts": {
"test": "NODE_OPTIONS=--experimental-vm-modules npx jest --coverage"
},
"dependencies": {
"zx": "^7.1.1"
},
"devDependencies": {
"jest": "^29.4.3",
"ts-jest": "^29.0.5",
"tsx": "^3.12.3",
"typescript": "^4.9.5"
}
}
Script (.ts) files have this as the first line:
#!/usr/bin/env tsx
In runtime environment (It is actually a docker container) I have zx, tsx, and tslib installed as global packages:
npm i --no-audit --no-fund -g zx tsx tslib
There's no need to copy or install dependencies into runtime environment. Just need to copy those script (.ts) files.
This is the content of tsconfig.json (*.ts script files are under scripts/ directory):
{
"extends": "../tsconfig-base.json",
"include": ["scripts/**/*"],
"compilerOptions": {
"target": "ES2022",
"module": "ES2022",
"moduleResolution": "node",
"lib": ["ES2021"],
"skipLibCheck": true,
"outDir": "dist"
}
}
This is the content of jest.config.cjs:
/** @type {import('ts-jest').JestConfigWithTsJest} */
module.exports = {
preset: 'ts-jest/presets/default-esm',
extensionsToTreatAsEsm: ['.ts'],
testEnvironment: 'node',
transform: {
'^.+\\.(ts|tsx)?$': ['ts-jest', { useESM: true }],
},
roots: ['scripts', 'test'],
collectCoverageFrom: ['scripts/**/*.ts'],
coverageThreshold: {
global: {
lines: 30,
},
},
};
In test files (*.test.ts) I have:
import { describe, it, expect } from '@jest/globals';
I believe this setup works by
- Running test cases with locally installed zx and other dependencies through locally installed ts-jest+jest
- So that we don't need special setup for local development environment or CI environment
- Running script files with globally installed zx and tslib through globally installed tsx
- So that there's no need to build-then-deploy, just need to copy or modify those script (.ts) files directly
What I have not setup for is other 3rd party dependencies (e.g. lodash).
- Maybe those dependencies can be installed globally?
- Or is it actually easier if all dependencies (including tslib and zx) in local
node_modules/are copied into the runtime environment?
And I did notice the startup delay (>5s) caused by tsx starting up.
If disk space and build/deployment time is not an issue, I think an alternative approach is to build executable pre-compiled bundles using tools like pkg. That way, the startup time would be close to zero, and there would be no requirement for the runtime environment. But that also means one huge (70MB for example) executable file per script.
You can also install tsx as a dependancy and call:
foo.ts:
#!/usr/bin/env -S node_modules/.bin/tsx
import { argv } from 'zx';
console.log('foo', argv);
``
```sh
chmod +x foo.ts
./foo
``
You can also install
tsxas a dependancy and call:
foo.ts:#!/usr/bin/env -S node_modules/.bin/tsx import { argv } from 'zx'; console.log('foo', argv); ``
We also recommend vite-node now, which implements more useful functions based on vite
Since bun is now "stable" you might want to use this kind of shebang:
#! /usr/bin/env -S bun run
import "zx/globals"
await $`ls -la`
Very very fast compared to any other alternatives right now, but you might encounter some issues as bun is still early.
$ cat dd.ts
#! /usr/bin/env -S bun run
import "zx/globals"
await $`/bin/ls -la`
$ time ./dd.ts &> /dev/null
./dd.ts &> /dev/null 0.16s user 0.11s system 151% cpu 0.179 total
$ time /bin/ls -la &> /dev/null
/bin/ls -la &> /dev/null 0.01s user 0.00s system 93% cpu 0.013 total
I wanted to run my script with tsx myScript.ts.
And I wanted to be able to have my files start like this
import { fileURLToPath } from 'node:url'
import { cd, chalk, fs, question, $ } from 'zx'
// ... rest of script ...
I was getting
Cannot find module 'zx'. Did you mean to set the 'moduleResolution' option to 'nodenext', or to add aliases to the 'paths' option?ts(2792)
So I set "moduleResolution": "NodeNext" in tsconfig.json, but then I got this error
The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is an ECMAScript module and cannot be imported with 'require'. Consider writing a dynamic 'import("zx")' call instead.
To convert this file to an ECMAScript module, change its file extension to '.mts' or create a local package.json file with `{ "type": "module" }`.ts(1479)
To fix that I added "type": "module" to my package.json file.
And now it worked. So there are two files you potentially need to change to make it all work, package.json and tsconfig.json.
With "type": "module" in package.json and this tsconfig.json VSCode no longer complains
{
"extends": "@tsconfig/recommended/tsconfig.json",
"compilerOptions": {
"module": "NodeNext",
"target": "ESNext"
}
}
Thanks @mathix420, that solution was super easy to implement for me. I literally just install bun, change the shebang, and add import zx/globals, and it works.
A minor issue is that VS Code complains that top-level await is not allowed, but it runs fine.
Adding this to tsconfig.json worked:
"target": "es2017",
"module": "es2022",
I then got into a bit more whack-a-mole with VS Code warnings and ended up adding these two as well:
"moduleResolution": "nodenext",
"allowImportingTsExtensions": true,
and importing my other TS modules with their .ts extension.