Shell commands?

[benbenbenbenbenben]

How do I run shell commands? I expected it to work like the python notepads do i.e.

! npm install some-package

Any advice?

[yingshaoxo]

I think this is very important since some of the users run this notebook on a separate virtual machine.

!npm i @tensorflow/tfjs

If python notebook can do this, why javascript can't?

[athewsey]

+1 to this request, and +1 to my main use case for it being installation of packages from the notebook (and even TF.js! 😂)

I would add that there is some precedent to other-language kernels not supporting this: E.g. I think the IRKernel project has declined to implement 'magics' multiple times as a design decision - leaving users stuck running shell commands via R's built-in methods.

...But the ! line magic in IPython is super helpful so would love to see it added to tslab!

For now, I guess I'll stick with

const { execSync } = require("child_process");
execSync("npm install @tensorflow/tfjs", { encoding: "utf-8" });

[yunabe]

Thank you for the feedbacks.

I'm a little bit skeptical about the idea to add ! command syntax to TypeScript/JavaScript kernels because ! is a valid operation in JavaScript.

let ls = true;
! ls  // false in JavaScript

What are the advantages of ! line magic over libraries in child_process like spawn?

[athewsey]

Agree there are credible use cases for just writing !myvar (especially e.g. as the last line in a cell to just display a result) - so ! is probably not so good...

IMO the value, especially for shell commands, is twofold:

1. I'd argue the child_process/spawn approach is kind of not an obvious thing to do for many folks?
   • Even for a lot of JS/TS devs, spawning processes is not a super common requirement - so forgetful people like me would end up having to look it up again quite often
   • ipykernel veterans might be used to the idea of inline shell code and expect an easy way to do it; while inexperienced notebook users might not even think of it as a pattern
2. Streaming output is super useful and I would argue non-trivial

A simple execSync() as I used above is OK if you're happy waiting to completion for outputs, but what about longer-running processes where it's important to see intermediate results? E.g. slow running docker builds, or even a model training job that you had to kick off as a script for some reason?

I actually had experience of this on a brief foray into IRKernel a while ago: Even after I'd got used to using system2(), having to just sit staring at the cell wondering how far a long docker build job had progressed (or whether I needed to kill it) was pretty frustrating... And I could never figure out a way to pipe stdout and stderr live back to the notebook in a way it could deal with - for a more IPython-like experience.

If ! doesn't translate well to JS, maybe tslab could still adopt the leading %/%% reservation for cell and line magics?

As far as I know, % is always a binary operator in JS/TS so there's really no reason to start a line with it... And this way, tslab could adopt the %%bash and/or %%sh (maybe even the %%script parent they're shortcuts to) magic for whole cells like:

%%sh
npm install @tensorflow/tfjs

...Or maybe consider adding JS-friendly line magics to replace the !, like below?

%sh npm install @tensorflow/tfjs

[nashspence]

@yunabe what spawn implementation should work? I have been trying:

` import { spawn } from "child_process";

async function sh(cmd) { return new Promise((resolve) => { const split = cmd.split(" "); const [command, ...args] = split; const sp = spawn(command, args, { cwd: process.cwd(), stdio: "pipe", shell: true }); sp.stdout.on("data", data => console.log(data.toString())); sp.stderr.on("data", data => console.error(data.toString()));

    sp.on("close", (code) => {
        console.log(`child process exited with code ${code}`);
        resolve();
    });
}); 

}; `

This keeps giving me errors:

unexpected error: Error: Unexpected pending rebuildTimer at Object.sys.setTimeout (/Users/nashspence/Desktop/boop/node_modules/tslab/dist/converter.js:83:19) at scheduleProgramUpdate (/Users/nashspence/Desktop/boop/node_modules/@tslab/typescript-for-tslab/lib/typescript.js:111417:41) at onSourceFileChange (/Users/nashspence/Desktop/boop/node_modules/@tslab/typescript-for-tslab/lib/typescript.js:111491:13) at /Users/nashspence/Desktop/boop/node_modules/@tslab/typescript-for-tslab/lib/typescript.js:104579:78 at updateContent (/Users/nashspence/Desktop/boop/node_modules/tslab/dist/converter.js:568:9) at Object.convert (/Users/nashspence/Desktop/boop/node_modules/tslab/dist/converter.js:218:9) at Object.execute (/Users/nashspence/Desktop/boop/node_modules/tslab/dist/executor.js:140:38) at JupyterHandlerImpl.handleExecuteImpl (/Users/nashspence/Desktop/boop/node_modules/tslab/dist/jupyter.js:219:38) at /Users/nashspence/Desktop/boop/node_modules/tslab/dist/jupyter.js:177:57 at async JupyterHandlerImpl.handleExecute (/Users/nashspence/Desktop/boop/node_modules/tslab/dist/jupyter.js:177:21)