CLI: toggle JS/TS output

[samijaber]

We want to add some logic to toggle TS output on/off in the CLI commands, and select .ts/.js (or .jsx/.tsx) based on that. To do that:

• have the core generators always emit TS, and add a MitosisConfig["outputs"]: ["typescript" | "javascript"] flag to the CLI.
  • If the outputs array contains typescript, we output the TS code as-is.
  • If the outputs array contains javascript, an additional transpilation step is executed that takes the TS code and transpiles it down to JS, and saves that to a file

The outputs array can contain 1 or both of those flags. If it's empty, we default to typescript 🤔.

PS: We can probably reuse this function for transpilation: https://github.com/BuilderIO/mitosis/blob/ab0708fd2c07a06a7a52ab7490cd8e716759a48a/packages/cli/src/build/helpers/transpile.ts#L17

Originally posted by @samijaber in https://github.com/BuilderIO/mitosis/issues/473#issuecomment-1154248996

[kylecordes]

Based on conversation @samijaber and I had elsewhere, as I understand the goal of Mitosis is to produce output in a form that it is ready to publish with npm and consume via node_modules. That is great, exactly what many projects (including mine...) want!

With TypeScript code, that generally means .js + .d.ts files. Projects very rarely expect to consume .ts files from node_modules. This seems to be "common knowledge" among mainstream TypeScript library authors, though I don't see anything from the TypeScript organization explicitly saying this.

I suspect Mitosis should emit:

• ts, to a directory not intended for publishing (.npmignore), for human inspection
• .js and .d.ts suitable for publishing/consumption

JavaScript-only project can readily consume this, seeing only the .js files. Their IDE will likely notice the typing files and offer better JS language support. For users who don't want any non-JS files (including not making types available downstream), a flag could trigger deleting all but the JS from the output.

[samijaber]

@kylecordes You're right that NPM packages are never published as .ts files!

It's worth noting that we also don't want to prematurely make assumptions about use cases. Our own usage at Builder so far has been to publish the Mitosis outputs as NPM modules, but it's very possible that folks out there will want something different! Perhaps some wild scenarios where the Mitosis output is generated as plain TS and then immediately consumed by sibling workspace projects, very much like how e2e app scaffold you've put together works, but with real-world production applications.

[samijaber]

Thought through this a bit more:

I suspect Mitosis should emit:

• ts, to a directory not intended for publishing (.npmignore), for human inspection
• .js and .d.ts suitable for publishing/consumption

For Mitosis to emit .js + .d.ts files, it would need to run tsc, which means that it would need to be provided a tsconfig.json for each output. Also, since mitosis build works by building & emitting files one at a time, this step would have to run at the very end, once all of the TS files have been generated and emitted. That's required for type-checking to make sense and look at the entire project.

So from looking at the above:

• The TS output is always needed as a precursor to the JS+types output (we could make it a temp output and delete it, if the user doesn't want it at the end)
• We will compile TS -> JS+types using tsc, which at this point is a completely separate step after mitosis build, wholly independent of the current mitosis build architecture/logic. All it does is take the output directories containing .ts and run tsc on them. It might as well be a new command.

The one challenge/question becomes: is it possible for Mitosis to figure out what the tsconfig.json should be for each output? Each user might want to target a different JS output...at that point, I wonder if it's worth it to bake this logic into Mitosis. I'm envisioning some separate mitosis build-tsc command (which can be run under-the-hood at the end of mitosis build --compile-js.

Pros:

• end-to-end, zero-config experience for our users looking to generate JS+types (which is likely the most common path!)

Cons:

• complex work for us to figure out all the correct configurations for every tsconfig.json. It will have to also be derived from the original tsconfig.json configuration of the Mitosis source code, with the necessary changes made to work with the final project. I foresee tons of trouble and edge-cases
• If users have a workflow that doesn't fit what we configured, they'll have to output TS and compile it down to JS on their own anyways.

Cons outway the pros for me right now. I do think it's a complex can of worms that I want to punt until we have 10+ real-world projects, building with Mitosis, and compiling the TS down to JS+.d.ts. Then we'll have enough data points and collect feedback. If we see a very common set of use-cases, we can abstract that out into the CLI.

Recap

There are 3 possible outputs

• plain .js (currently supported)
• .ts (supported in some generators, possible in all)
• .js+.d.ts (requires additional work and machinery)

[kylecordes]

There is obvious good wisdom in punting on the complexity. That would mean Mitosis output is always "source”, never files ready to consume direct as a library. The user would then arrange to compile the .*.ts files as desired, in the source side of their application. (As far as I can tell from years working with TypeScript, tooling doesn’t want to consume .ts files from node_modules. It can sometimes be made to work, but is “out of spec”.)

Your analysis is good, but I think the TS-library-output mechanism is not nearly as hard it is seems. There are tens of thousands (hundreds of thousands?) of TS-based libs on npm already, that ship js and d.ts. The key bit is: a library uses its own tsconfig; library consumers don't need a library to change the library's tsconfig.

Modern libraries sometimes emit both down-leveled (for easy Node consumption) and module JavaScript at ES2018 or ES2020 - a consuming application can use whatever tooling it wants, to get other JS variants from that if needed. Consuming libraries either use the d.ts files or ignore them.

It would be great if @mhevery could weigh in, likely with more expertise than I could dream of. (Angular studied all this in great detail and arrived at a very robust process and package output spec.)

[samijaber]

It could be simpler than I am anticipating! We will soon switch the Builder SDKs to the ts output, and then build js+.d.ts files from that so that the final result has types. If I see that the work needed there is easily portable, we can definitely bake it into the CLI.

[samijaber]

Thinking out loud some more. Another set of Mitosis-specific concerns I realized we will encounter is:

Say I have Mitosis source-code that I compile to React. If the tsc command is to run at the end of mitosis build, then this means that Mitosis needs to be aware of, and make a choice regarding which versions of the framework to provide: react/react-dom need to be part of this typescript project (otherwise type-checking will fail). So this means that mitosis build now needs to:

• generate .tsx files in output/react/** representing the Mitosis output
• add a package.json
• ask the user to pick a React version, along with any other dependencies, like styled-jsx which we rely on for css-in-js if the user chooses so. We do this a bunch actually: relying on popular 3rd party libs to handle certain aspects of component logic.
• install all that using npm/yarn, and hope it passes
• run tsc

If you do that for 3-4 frameworks, Mitosis is now handling quite a bit of logic around building TS projects. And if there's an issue at any step of the way (in any target output), the complexity around error-handling or fallback logic will grow quickly.

This work could theoretically be in some other command, e.g. mitosis add react@16: it would add the entry to mitosis.config.js, create the required output/react/package.json and run any installation steps. It's all stuff that we have to do as end-users right now anyways, and that Mitosis could eventually take care of for us. 🤔

[kylecordes]

Yes, exactly. I definitely don't urge growing scope too fast... but it would be perfect if eventually the config/flags included telling Mitosis what version of React/etc to use, then Mitosis would emit the package.json and all other needed files, ready to use with no further edits!

[samijaber]

Closing as this has been fully implemented now!

[dylan-reniers-digitalum]

Perhaps it would be nice to have some documentation on how to properly get the typescript output to work?