zx
zx copied to clipboard
Improve README documentation for TypeScript usage
Expected Behavior
The 'TypeScript' section of the README file would provide accurate, complete, and up to date information of how to use zx
with TypeScript.
Actual Behavior
The current 'how to use TypeScript' documentation appears inadequate:
https://github.com/google/zx/blob/854e4eca9e42f056fa9d1de9f5e7abd1e7461cbd/README.md?plain=1#L454-L468
Crawling through the release history it seems like support for TypeScript via the zx
binary was dropped in v5.0.0:
- https://github.com/google/zx/releases/tag/5.0.0
This release drops build of CommonJS version and support for
.ts
extension byzx
bin.TypeScript is still supported, for example, via ts-node:
node --loader ts-node/esm script.ts
Also, a new Node version requirement is >= 16.0.0.
Looking at the README at this commit seemed to have better usage information for how to use this with ts-node
:
https://github.com/google/zx/blob/7977cb5adb9545082b8edb8bfe647dedfcb98b42/README.md?plain=1#L398-L421
Which seems to have been removed ~17 days ago in this commit: https://github.com/google/zx/commit/db0e65163d31e37cc6d71ae2e20e2ada4186efa6#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L542-R458
Following the current README as written, I tried making a foo.mjs
test file:
#!/usr/bin/env zx
import 'zx/globals'
void async function () {
const foo: string = "foo";
await $`ls -la`
}()
Running this with ./foo.mjs
(zx
bin in the shebang) results in the following error, presumably because .mjs
doesn't specify it as a TypeScript file:
SyntaxError: Missing initializer in const declaration
at ESMLoader.moduleStrategy (node:internal/modules/esm/translators:117:18)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:337:14)
at async link (node:internal/modules/esm/module_job:70:21)
I also tried renaming it to foo.ts
, which resulted in this error:
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /Users/devalias/dev/0xdevalias/minimal-zx-ts/foo.ts
at new NodeError (node:internal/errors:372:5)
at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:76:11)
at defaultGetFormat (node:internal/modules/esm/get_format:118:38)
at defaultLoad (node:internal/modules/esm/load:21:20)
at ESMLoader.load (node:internal/modules/esm/loader:407:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:326:22)
at new ModuleJob (node:internal/modules/esm/module_job:66:26)
at ESMLoader.#createModuleJob (node:internal/modules/esm/loader:345:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:304:34)
at async Promise.all (index 0) {
code: 'ERR_UNKNOWN_FILE_EXTENSION'
}
Looking at the TypeScript docs for 'ECMAScript Modules in Node.js' it talks about various other file extensions that can be used:
- https://www.typescriptlang.org/docs/handbook/esm-node.html#new-file-extensions
The type field in package.json is nice because it allows us to continue using the
.ts
and.js
file extensions which can be convenient; however, you will occasionally need to write a file that differs from what type specifies. You might also just prefer to always be explicit.Node.js supports two extensions to help with this:
.mjs
and.cjs
..mjs
files are always ES modules, and.cjs
files are always CommonJS modules, and there’s no way to override these.In turn, TypeScript supports two new source file extensions:
.mts
and.cts
. When TypeScript emits these to JavaScript files, it will emit them to .mjs
and.cjs
respectively.
Based on this, I figured that maybe the .mts
extension would work, as it should be the 'TypeScript flavoured' version of the recommended .mjs
; but renaming to foo.mts
and running it again resulted in this error:
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".mts" for /Users/devalias/dev/0xdevalias/minimal-zx-ts/foo.mts
at new NodeError (node:internal/errors:372:5)
at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:76:11)
at defaultGetFormat (node:internal/modules/esm/get_format:118:38)
at defaultLoad (node:internal/modules/esm/load:21:20)
at ESMLoader.load (node:internal/modules/esm/loader:407:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:326:22)
at new ModuleJob (node:internal/modules/esm/module_job:66:26)
at ESMLoader.#createModuleJob (node:internal/modules/esm/loader:345:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:304:34)
at async Promise.all (index 0) {
code: 'ERR_UNKNOWN_FILE_EXTENSION'
}
Based on the old README example of using ts-node
I decided to try that method, and followed the latest instructions/details from:
- https://github.com/TypeStrong/ts-node#native-ecmascript-modules
- https://github.com/TypeStrong/ts-node/issues/1007#issue-598417180
- https://github.com/TypeStrong/ts-node/issues/1007#issuecomment-1139917958
- https://github.com/TypeStrong/ts-node/issues/1007#issuecomment-1163471306
I can seemingly run the script with ts-node-esm
directly as:
npm exec ts-node-esm -- ./foo.mts
And by modifying the shebang to call ts-node-esm
:
#!/usr/bin/env npx --package=ts-node -- ts-node-esm
import 'zx/globals'
void async function () {
const foo: string = "foo";
await $`ls -la`
}()
We can run the file with a .ts
extension (when "type": "module",
is set in package.json
).
If we try to use a .js
or .mjs
extension we get the same error as earlier (as we would expect):
SyntaxError: Missing initializer in const declaration
at ESMLoader.moduleStrategy (node:internal/modules/esm/translators:117:18)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:337:14)
at async link (node:internal/modules/esm/module_job:70:21)
If we use the .mts
extension however, we can remove "type": "module",
from package.json
and it will still work (as .mts
are files are always ES modules)
We can make this run even faster if we use ts-node
's swc
suport:
- https://typestrong.org/ts-node/docs/transpilers/
-
ts-node
supports third-party transpilers as plugins. Transpilers such asswc
can transform TypeScript into JavaScript much faster than the TypeScript compiler. You will still benefit fromts-node
's automatictsconfig.json
discovery, sourcemap support, and globalts-node
CLI.
-
- https://typestrong.org/ts-node/docs/swc
-
SWC support is built-in via the
--swc
flag or"swc": true
tsconfig
option. -
SWC is a TypeScript-compatible transpiler implemented in Rust. This makes it an order of magnitude faster than vanilla
transpileOnly
. -
To use it, first install
@swc/core
or@swc/wasm
. If usingimportHelpers
, also install@swc/helpers
. If target is less than"es2015"
and usingasync
/await
or generator functions, also installregenerator-runtime
.
-
- https://github.com/swc-project/swc
- https://swc.rs/
Based on this we could install the @swc/core
lib, and modify our script's shebang to add the --swc
arg as follows:
npm i --save-dev @swc/core
#!/usr/bin/env npx --package=ts-node -- ts-node-esm --swc
import 'zx/globals'
void async function () {
const foo: string = "foo";
await $`ls -la`
}()
Super basic testing on my laptop with time
shows the following speed improvements when using --swc
with ts-node
:
// time ./foo.mts
// Without --swc
./foo.mts 3.45s user 0.55s system 126% cpu 3.151 total
./foo.mts 3.29s user 0.34s system 153% cpu 2.374 total
./foo.mts 3.16s user 0.34s system 156% cpu 2.243 total
// With --swc
./foo.mts 1.14s user 0.46s system 69% cpu 2.299 total
./foo.mts 1.14s user 0.27s system 103% cpu 1.358 total
./foo.mts 1.13s user 0.27s system 103% cpu 1.356 total
Contrasting this against a plain .mjs
file using the zx
bin directly, we can see that the .mjs
is still faster, but for the dev efficiency improvements of using TypeScript, I feel like the tradeoff isn't too bad when using tas-node
with --swc
:
#!/usr/bin/env zx
import 'zx/globals'
const foo = "foo";
await $`ls -la`
console.log(foo);
./foo.mjs 0.46s user 0.25s system 72% cpu 0.977 total
./foo.mjs 0.45s user 0.11s system 118% cpu 0.481 total
./foo.mjs 0.44s user 0.11s system 119% cpu 0.461 total
Trying to remove the wrapping async function
results in the following error:
/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:843
return new TSError(diagnosticText, diagnosticCodes, diagnostics);
^
TSError: ⨯ Unable to compile TypeScript:
foo.mts:7:3 - error TS1378: Top-level 'await' expressions are only allowed when the 'module' option is set to 'es2022', 'esnext', 'system', 'node16', or 'nodenext', and the 'target' option is set to 'es2017' or higher.
7 await $`ls -la`
~~~~~
at createTSError (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:843:12)
at reportTSError (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:847:19)
at getOutput (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:1057:36)
at Object.compile (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/index.ts:1411:41)
at transformSource (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/esm.ts:400:37)
at /Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/esm.ts:278:53
at async addShortCircuitFlag (/Users/devalias/.npm/_npx/1bf7c3c15bf47d04/node_modules/ts-node/src/esm.ts:409:15)
at async ESMLoader.load (node:internal/modules/esm/loader:407:20)
at async ESMLoader.moduleProvider (node:internal/modules/esm/loader:326:11)
at async link (node:internal/modules/esm/module_job:70:21) {
diagnosticCodes: [ 1378 ]
}
Which we can fix by setting our tsconfig.json
to something like this:
{
"compilerOptions": {
"target": "ES2021",
"module": "node16",
},
}
Which then allows us to simplify the basic code to:
#!/usr/bin/env npx --package=ts-node -- ts-node-esm
import 'zx/globals'
const foo: string = "foo";
await $`ls -la`
We could also use something like the following (from https://github.com/tsconfig/bases#node-16--esm--strictest-tsconfigjson) rather than handcrafting our own compatible tsconfig.json
:
npm i @tsconfig/node16-strictest-esm
// tsconfig.json
{
"extends": "@tsconfig/node16-strictest-esm/tsconfig.json"
}
An alternative to using ts-node
could be to use tsm
(which wraps esbuild
), but I didn't look too deeply into how to make it work:
- https://github.com/lukeed/tsm
- https://github.com/lukeed/tsm/blob/master/docs/usage.md#shell--shebang
- https://github.com/evanw/esbuild
Steps to Reproduce the Problem
- Be a newbie to using
zx
- Wonder how to use it effectively with TypeScript
- Get confused
Specifications
- Version: 7.0.5
- Platform: node v16.15.1, macOS 12.3.1
Other older TypeScript related issues that I looked through first
- https://github.com/google/zx/pull/409
- https://github.com/google/zx/issues/273
- https://github.com/google/zx/issues/206
- https://github.com/google/zx/issues/205
- https://github.com/google/zx/issues/197
- https://github.com/google/zx/issues/194
- https://github.com/google/zx/issues/152
- https://github.com/google/zx/issues/125
- https://github.com/google/zx/pull/114
- https://github.com/google/zx/issues/110
- https://github.com/google/zx/issues/75
Also potentially relevant:
I haven't gone far with this yet, but this is the shebang I'm using in a script to get
ts-node
to use ESM mode:#!/usr/bin/env -S npx ts-node --esm --compilerOptions={"module":"ESNext","target":"ESNext","moduleResolution":"node"} import { $ } from 'zx'; await $`echo ${'stuff'}`;
Originally posted by @tvsbrent in https://github.com/google/zx/issues/125#issuecomment-1169890068
Awesome! Lets add condensed end version of this to the readme.
I tried tsup, and it works for me. my tsup.config.json like this:
{
"entry": ["src/index.ts"],
"splitting": true,
"outDir": "bin",
"sourcemap": true,
"format": "esm",
"target": "node16",
"clean": true
}
packgae.json
"scripts": {
"dev": "tsup --watch"
}
I almost managed to marry zx with ts-node like this:
- install ts-node and zx globally
- create
foo.mts
(module ts file) with content:
#!/usr/bin/env -S ts-node-esm --compilerOptions='{"module": "es2022"}'
import 'zx/globals';
await $`echo it works ${JSON.stringify(foo(1))}`;
function foo(num: number): number[] {
return [num, num * 2, num * 3];
}
- execute "./foo.mts"
But the problem I'm facing is that by default, Node doesn't resolve import 'zx/globals'
to /usr/lib/node_modules/
.
Typescript fails with Error TS2592: Cannot find name '$'
.
If I install zx
in ./node_modules
, it works. But I want to avoid this.
I think currently zx + ts wotks only for locally installed dependencies. :₽
But will be cool if zx also supports global ts scripts.
Also potentially relevant:
I haven't gone far with this yet, but this is the shebang I'm using in a script to get
ts-node
to use ESM mode:#!/usr/bin/env -S npx ts-node --esm --compilerOptions={"module":"ESNext","target":"ESNext","moduleResolution":"node"} import { $ } from 'zx'; await $`echo ${'stuff'}`;
Originally posted by @tvsbrent in #125 (comment)
An update to what was referenced above. I have used the following shebang for a few different scripts now, and it seems to work pretty well:
#!/usr/bin/env -S npx ts-node --esm --project=./tsconfig.build.json
In this case, my tsconfig.build.json
has the following:
{
"extends": "./tsconfig.json",
"exclude": [
"node_modules",
"**/*.test.ts",
"**/*.spec.ts"
],
}
In the base tsconfig.json
, I have the following properties:
"target": "ESNext",
"module": "ESNext",
"moduleResolution": "node",
In addition, in the package.json
, I have set "type": "module"
.
Just published a quick helper script to make this easier. See repo
Install it npm install -g tzx
Use it tzx <script.ts>
Hey guys, I've got a nicely working solution for now. Requirements:
- package.json file that has
"type": "module"
-
npm install -g zx tsup
Create your script.ts file in the same directory as your package.json, example
#!/usr/bin/env zx
import { $ } from "zx";
await $`echo "test"`;
Now this is for linux only but you can adapt it to other platforms. Inside your .bashrc
add the following function:
zxe() {
INPUT=$1
OUTPUT=$(sed 's/.ts/.js/' <<< "$INPUT")
tsup-node -d . --format esm --target {{NODE_VERSION}} "$INPUT"
sed '/^import.*zx"/d' -i "$OUTPUT"
zx "$OUTPUT"
rm "$OUTPUT"
}
Replace {{NODE_VERSION}} with node18
or node16
or whatever.
Restart your terminal or you can run source ~/.bashrc
.
Now you can run your scripts with zxe script.ts
all other problems problems like, intelisense and imports work with this settings https://github.com/felipeplets/esm-examples
I've found success using the .mts
extension and adding the following shebang:
#!/usr/bin/env -S npx tsx
import 'zx/globals';
await $`ls -la`;
console.log(chalk.green('Hello, World!'));
Run like so:
chmod +x ./myfile.mts
./myfile.mts
Just published a quick helper script to make this easier. See repo
Install it
npm install -g tzx
Use ittzx <script.ts>
Hey guys, I've got a nicely working solution for now. Requirements:
- package.json file that has
"type": "module"
npm install -g zx tsup
Create your script.ts file in the same directory as your package.json, example
#!/usr/bin/env zx import { $ } from "zx"; await $`echo "test"`;
Now this is for linux only but you can adapt it to other platforms. Inside your
.bashrc
add the following function:zxe() { INPUT=$1 OUTPUT=$(sed 's/.ts/.js/' <<< "$INPUT") tsup-node -d . --format esm --target {{NODE_VERSION}} "$INPUT" sed '/^import.*zx"/d' -i "$OUTPUT" zx "$OUTPUT" rm "$OUTPUT" }
Replace {{NODE_VERSION}} with
node18
ornode16
or whatever.Restart your terminal or you can run
source ~/.bashrc
.Now you can run your scripts with
zxe script.ts
Your lib worked great for me, thanks!
I've tried most of these suggestions, but I keep getting stumped by $
not being recognized
e.g.
#!/usr/bin/env npx --package=ts-node -- ts-node-esm
import "zx/globals";
await $`echo ${"stuff"}`;
Results in
error TS2592: Cannot find name '$'. Do you need to install type definitions for jQuery? Try `npm i --save-dev @types/jquery`
I do have zx globally installed.
Why not use tsx instead? It's based on esbuild instead of typescript, can compile ts but doesn't check types, is fast and great for scripts. In addition, it also supports esm/cjs, replacing esmo/esno. ref: https://github.com/esbuild-kit/tsx
Why not use tsx instead? It's based on esbuild instead of typescript, can compile ts but doesn't check types, is fast and great for scripts. In addition, it also supports esm/cjs, replacing esmo/esno. ref: https://github.com/esbuild-kit/tsx
It's the $
feature that zx
provides that makes it so powerful. The ability to write env vars back to the shell is something that is quite non-trivial without a tool like zx
, if tsx
support that too then I wouldn't mind switching but I doubt it's built for that use case.
Why not use tsx instead? It's based on esbuild instead of typescript, can compile ts but doesn't check types, is fast and great for scripts. In addition, it also supports esm/cjs, replacing esmo/esno. ref: https://github.com/esbuild-kit/tsx
It's the
$
feature thatzx
provides that makes it so powerful. The ability to write env vars back to the shell is something that is quite non-trivial without a tool likezx
, iftsx
support that too then I wouldn't mind switching but I doubt it's built for that use case.
you just need
import 'zx/globals';
Here's my setup with tsx...
Create a bin directory & create a node_modules there for the script environment...
mkdir ~/bin
cd ~/bin
npm install zx
cd ..
Create a small test script...
cat > ~/bin/foo.ts
#!/usr/bin/env tsx
import 'zx/globals';
$`echo hello`;
<ctl-d>
Now to test the test script...
~/bin/foo.ts
This outputs:
$ echo hello
hello
Maybe something like this can be added to the README?
I recently encountered the same TypeError [ERR_UNKNOWN_FILE_EXTENSION]
error. To solve this I added shebang and run the file with ts-node
.
Here's the example of how I did this:
run.zx.ts
#!/usr/bin/env zx
import { $ } from 'zx/core'
await $`echo "hi"`
package.json
{
"scripts": {
"zx": "ts-node run.zx.ts"
}
}
With this setup I was able to execute the script using zx
without any errors.
How about executing with deno instead of node?
Just to share my working configuration. My requirements are:
- *.ts files can be executed directly
- I can run test cases with jest
- The Node.js version is 18
This is what I have in package.json
(only related parts are shown):
{
"type": "module",
"engines": {
"node": ">=18.0.0"
},
"scripts": {
"test": "NODE_OPTIONS=--experimental-vm-modules npx jest --coverage"
},
"dependencies": {
"zx": "^7.1.1"
},
"devDependencies": {
"jest": "^29.4.3",
"ts-jest": "^29.0.5",
"tsx": "^3.12.3",
"typescript": "^4.9.5"
}
}
Script (.ts) files have this as the first line:
#!/usr/bin/env tsx
In runtime environment (It is actually a docker container) I have zx
, tsx
, and tslib
installed as global packages:
npm i --no-audit --no-fund -g zx tsx tslib
There's no need to copy or install dependencies into runtime environment. Just need to copy those script (.ts) files.
This is the content of tsconfig.json
(*.ts script files are under scripts/
directory):
{
"extends": "../tsconfig-base.json",
"include": ["scripts/**/*"],
"compilerOptions": {
"target": "ES2022",
"module": "ES2022",
"moduleResolution": "node",
"lib": ["ES2021"],
"skipLibCheck": true,
"outDir": "dist"
}
}
This is the content of jest.config.cjs
:
/** @type {import('ts-jest').JestConfigWithTsJest} */
module.exports = {
preset: 'ts-jest/presets/default-esm',
extensionsToTreatAsEsm: ['.ts'],
testEnvironment: 'node',
transform: {
'^.+\\.(ts|tsx)?$': ['ts-jest', { useESM: true }],
},
roots: ['scripts', 'test'],
collectCoverageFrom: ['scripts/**/*.ts'],
coverageThreshold: {
global: {
lines: 30,
},
},
};
In test files (*.test.ts) I have:
import { describe, it, expect } from '@jest/globals';
I believe this setup works by
- Running test cases with locally installed zx and other dependencies through locally installed ts-jest+jest
- So that we don't need special setup for local development environment or CI environment
- Running script files with globally installed zx and tslib through globally installed tsx
- So that there's no need to build-then-deploy, just need to copy or modify those script (.ts) files directly
What I have not setup for is other 3rd party dependencies (e.g. lodash).
- Maybe those dependencies can be installed globally?
- Or is it actually easier if all dependencies (including tslib and zx) in local
node_modules/
are copied into the runtime environment?
And I did notice the startup delay (>5s) caused by tsx starting up.
If disk space and build/deployment time is not an issue, I think an alternative approach is to build executable pre-compiled bundles using tools like pkg. That way, the startup time would be close to zero, and there would be no requirement for the runtime environment. But that also means one huge (70MB for example) executable file per script.
You can also install tsx
as a dependancy and call:
foo.ts
:
#!/usr/bin/env -S node_modules/.bin/tsx
import { argv } from 'zx';
console.log('foo', argv);
``
```sh
chmod +x foo.ts
./foo
``
You can also install
tsx
as a dependancy and call:
foo.ts
:#!/usr/bin/env -S node_modules/.bin/tsx import { argv } from 'zx'; console.log('foo', argv); ``
We also recommend vite-node now, which implements more useful functions based on vite
Since bun is now "stable" you might want to use this kind of shebang:
#! /usr/bin/env -S bun run
import "zx/globals"
await $`ls -la`
Very very fast compared to any other alternatives right now, but you might encounter some issues as bun
is still early.
$ cat dd.ts
#! /usr/bin/env -S bun run
import "zx/globals"
await $`/bin/ls -la`
$ time ./dd.ts &> /dev/null
./dd.ts &> /dev/null 0.16s user 0.11s system 151% cpu 0.179 total
$ time /bin/ls -la &> /dev/null
/bin/ls -la &> /dev/null 0.01s user 0.00s system 93% cpu 0.013 total
I wanted to run my script with tsx myScript.ts
.
And I wanted to be able to have my files start like this
import { fileURLToPath } from 'node:url'
import { cd, chalk, fs, question, $ } from 'zx'
// ... rest of script ...
I was getting
Cannot find module 'zx'. Did you mean to set the 'moduleResolution' option to 'nodenext', or to add aliases to the 'paths' option?ts(2792)
So I set "moduleResolution": "NodeNext"
in tsconfig.json
, but then I got this error
The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is an ECMAScript module and cannot be imported with 'require'. Consider writing a dynamic 'import("zx")' call instead.
To convert this file to an ECMAScript module, change its file extension to '.mts' or create a local package.json file with `{ "type": "module" }`.ts(1479)
To fix that I added "type": "module"
to my package.json
file.
And now it worked. So there are two files you potentially need to change to make it all work, package.json
and tsconfig.json
.
With "type": "module"
in package.json
and this tsconfig.json
VSCode no longer complains
{
"extends": "@tsconfig/recommended/tsconfig.json",
"compilerOptions": {
"module": "NodeNext",
"target": "ESNext"
}
}
Thanks @mathix420, that solution was super easy to implement for me. I literally just install bun, change the shebang, and add import zx/globals
, and it works.
A minor issue is that VS Code complains that top-level await
is not allowed, but it runs fine.
Adding this to tsconfig.json
worked:
"target": "es2017",
"module": "es2022",
I then got into a bit more whack-a-mole with VS Code warnings and ended up adding these two as well:
"moduleResolution": "nodenext",
"allowImportingTsExtensions": true,
and importing my other TS modules with their .ts
extension.