Running a standalone script

[45930]

trafficstars

Describe the problem

It's common for someone looking to troubleshoot a problem to share a gist or some code in a single text file. Anyone looking to reproduce the issue needs to put that file into a zk project context, covert the typescript and run the javascript. This seems a) repetitive and b) prone to error / environment mismatches.

Describe the proposed solution

I think a nice feature would be an API that looks like zk run myScript.ts which would convert the file to javascript and run it. It would be runnable globally on a machine with the zkapp-cli installed such that a file could just be downloaded or tossed into a documents folder and be run.

Alternatives considered

No response

Importance

would make my life easier

Additional Information

This is sort of a temperature check. I'm happy to iterate with more detail if it's something that makes sense to exist in this project.

[mitschabaude]

my temperature is 🔥 it should also have --watch

[jasongitmail]

So it'd essentially be an alias for ts-node?

ts-node foo.ts
OR
npx ts-node foo.ts

[45930]

Nice, I didn't know about ts-node. I think the main value of either replicating or aliasing the behavior into zkapp-cli would be the consistency across 2+ computers. It would be the most easy way to ensure tsconfig (I was assuming we could just use the default from the example project) and snarkyjs versions are correct, and would work well with zk system.

[mitschabaude]

Exactly, there's the custom ts-config, plus a few niceties we can add like

• automatically resolve snarkyjs
• executing with a higher --stack-trace-limit to help with our humongous stack traces
• automatically call shutdown

[jasongitmail]

automatically resolve snarkyjs

hmm that's encouraging some pretty sloppy coding and ts should be yelling at the dev for not having this. But could be convenient.

executing with a higher --stack-trace-limit to help with our humongous stack traces

useful in the near term, possibly not long term

automatically call shutdown

a non-point imo b/c it won't be required soon hopefully after refactoring the workers

custom ts-config

the best argument

[mitschabaude]

hmm that's encouraging some pretty sloppy coding and ts should be yelling at the dev for not having this

that's true, ts / the linter wouldn't like it, so only useful in very quick-&-dirty scenarios; not important

[45930]

So I messed around a little bit with ts-node. Within a project it works as expected, but trying to run it from another directory gives me issues.

Documents~ NODE_OPTIONS='--stack-trace-limit=1000 --experimental-vm-modules --experimental-wasm-modules' ts-node-esm --project ../Projects/zkapps/zk-cli-project/tsconfig.json example.ts

Documents~ NODE_OPTIONS='--stack-trace-limit=1000 --experimental-vm-modules --experimental-wasm-modules' ts-node-esm --project ../Projects/zkapps/zk-cli-project/tsconfig.json -r /Users/path/.nvm/versions/node/v16.14.2/lib/node_modules/snarkyjs/dist/node/index.cjs example.ts

Various examples of this ^^ fail with

error TS2307: Cannot find module 'snarkyjs' or its corresponding type declarations.
1 import { isReady, Field, shutdown } from 'snarkyjs';

I ran npm i -g snarkyjs obviously.

Running from within a project directory works as expected:

NODE_OPTIONS='--stack-trace-limit=1000 --experimental-vm-modules --experimental-wasm-modules' ts-node-esm example.ts
10

example.ts is

import { isReady, Field, shutdown } from 'snarkyjs';
await isReady;
const f = Field(10);
console.log(f.toString());
shutdown();

Is this a known behavior about snarkyjs that it can't be imported globally?

[mitschabaude]

This is not known behavior to me, but I also haven't tried it. It would surprise me, though, if there was something special about snarkyjs here. Maybe it's some CJS vs ESM thing?

Btw, you don't need --experimental-wasm-modules, and probably also don't need --experimental-vm-modules (except if this is some ts-node thing I don't know about)

[mitschabaude]

@45930 I also tried some things now. there are several hurdles to overcome:

• TS doesn't resolve global packages. so it fails with a compile error. this is what you're seeing above through ts-node
• Even if you get past that (for example, disable type checking), there's the problem that node doesn't resolve global packages during import. That can be solved with an environment variable. Sadly, that only seems to work importing CJS modules (or at least I couldn't get it to work differently so far)
• ts-node / ts-node-esm seems to always rewrite the input to CJS so you can't use top level await. Because of the last point, it seems you have to rewrite to CJS anyway though, so to support top level await we'd need to wire up a toolchain that rewrites TLA to something CJS-compatible (meh)

So, here's a way to get it working (without top-level await, and with esbuild installed globally):

esbuild --format=cjs test.ts | NODE_PATH=$(npm root -g) node

transpiling with esbuild is fast and doesn't type-check. The NODE_PATH makes node resolve the globally installed snarkyjs.

perhaps we can build on that. probably needs to be adapted for windows? here's the file:

// test.ts
import { isReady, Field, shutdown } from "snarkyjs";

main();

async function main() {
  await isReady;
  console.log(Field(5));
  shutdown();
}

[mitschabaude]

I think a reasonable first iteration of zk run could be:

• tries to run it with ts-node-esm without overriding ts config (should work in zk-cli projects; we shouldn't override the config if one exists)
• if that fails, tries to run it with ts-node-esm + custom ts config (should work in any folder that has snarkyjs installed locally)
• if that fails, tries some non-type checked / less polished way like the above (also with custom config). should work everywhere

~~I think it would be important to get top level await working, at least for the first two cases. Maybe there's a way with ts-node to do that, or we find another similar tool that can do it. ts-node seems kind of tied to common JS which isn't great~~ EDIT: not an issue, see below

[mitschabaude]

Oh I just found the NODE_PATH trick works with ts-node as well, makes case 3 even simpler:

NODE_PATH=$(npm root -g) ts-node-esm -T test.ts

[mitschabaude]

Found out that in a folder with the right tsconfig and snarkyjs installed locally, top level await actually works:

ts-node-esm test.ts # with TLA

so the top level await limitation only remains in the global / execute anywhere case

[45930]

@mitschabaude thanks for looking into it.

here's the file:

It seems like any file with TLA could be converted by just wrapping it in

main();

async function main() {
  // input file contents here
}

I think case 3 is the only one that aligns with the original value prop. If we are already in a project with snarkyjs installed, then we can just run npm run build && node ... which seems more natural.

If you change the scope to allow for pulling the packages and config of the local project, what is the value of zk run (not a rhetorical question)? One thing that comes to mind is just to log some additional info like the snarkyjs version which would give some better starting point for looking into an issue.

Mainly I was envisioning a way for 2+ devs to troubleshoot an issue without cloning the same repository, a much lighter way to check something out if someone claims to have a bug or issue. Are you thinking the same thing? Or are you thinking of zk run as a devX feature which is easier to use for a single dev building something and running a script?

[mitschabaude]

I think case 3 is the only one that aligns with the original value prop. If we are already in a project with snarkyjs installed, then we can just run npm run build && node ... which seems more natural. If you change the scope to allow for pulling the packages and config of the local project, what is the value of zk run (not a rhetorical question)?

• better debugging (TS / OCaml paths in the stack traces, instead of JS)
• is shorter & simpler to type
• will have a --watch mode

Mainly I was envisioning a way for 2+ devs to troubleshoot an issue without cloning the same repository, a much lighter way to check something out if someone claims to have a bug or issue. Are you thinking the same thing? Or are you thinking of zk run as a devX feature which is easier to use for a single dev building something and running a script?

I was thinking DX for one person. But the use case for simpler debugging is cool as well!

[mitschabaude]

It seems like any file with TLA could be converted by just wrapping it in

I don't think it's that simple! wrapping in a main function would change the execution order when the file with TLA is imported

I actually would prefer to not mess with user's files, except if we use a well tested tool for making this transformation

[45930]

better debugging (TS / OCaml paths in the stack traces, instead of JS)

Yeah that checks out with me. I have more than once edited the js file in a stack trace, rebuilt, and wondered why nothing changed... 😂

I actually would prefer to not mess with user's files

I agree in the context of the zkapp-cli package, it's probably too hacky to be copying and manipulating files. It would be error prone across operating systems and architectures. But in theory it sounds very nice to run a target file within a project that just lives in .nvm/node16/zkapp-cli/dummy-project where we could copy the file to and run in the run-from-anywhere case.

I was thinking DX for one person

But the use case for simpler debugging is cool as well!

I think DX for one person case desires the run-from-project functionality, and the debugging case desires run-from-anywhere and I don't think there's actually very much overlap in the execution. For run-from-anywhere the value is that zkapp-cli ships with a default tsconfig and a target snarkyjs version with which to execute some anonymous file. For run-from-project, you don't want a default tsconfig or a default snarkyjs; you want the exact config of your project.

[45930]

If run-from-project existed, then it's a fairly simple exercise for someone to

zk p dummy
mv myfile.ts dummy
cd dummy
npm i && zk run myfile.ts

Which is run-from-anywhere with slightly more steps. That seems like an acceptable solution to the running an anonymous file use case given that ts-node doesn't seem to play nice with global packages.