[package-generator] New templates for package development

[savetheclocktower]

Have you checked for existing feature requests?

• [x] Completed

Summary

I've had it on my to-do list for a while to try to make a dent in two different problems for Pulsar community package development:

1. There's no canonical path for those who want to write their package in TypeScript. One can look around for, and find, other popular packages that are written in TypeScript — but their toolchains are many years old and probably not ideal to model a new package after.
2. It's an ongoing pain point that Electron’s renderer-process Node integration does not allow us to use ES modules; all package code must be written in, or compile down to, CommonJS. Since more and more modules are available only as ESM rather than offering a CommonJS equivalent, this leaves the package author with some distasteful options to choose between, including (a) transpiling ESM imports to CommonJS, or (b) using older versions of these libraries before they went ESM-only.

These are related issues; whatever I come up with for problem 2 must also be a part of problem 1.

I spent a few hours on this today; for posterity, here's what I've arrived at:

• Use rollup as the bundler for both templates
• Use @rollup/plugin-commonjs and @rollup/plugin-node-resolve in the config file
• Explicitly add /node_modules/ to the include option when configuring plugin-commonjs
• Add 'atom' to the list of external modules in the options so that Rollup doesn't complain about it when compiling
• Optionally, the user can add any other dependencies to external if they're already CommonJS (i.e., don't need transpiling)

In my testing, this was sufficient to transpile my ES Module code, including an ESM-only dependency.

I think it's a good idea to make package-generator presets for these use cases, and even to consider making the JS version the default for the command Package Generator: Generate Package. (The TypeScript preset can be made from a new command called Package Generator: Generate TypeScript Package (or something like that).

Adding @rollup/plugin-typescript introduces slightly more complexity, but wasn't too hard to do. It does seem to be finicky about exact values in tsconfig.json, especially for moduleResolution.

---

What follows are sample config files for this experiment.

The JS version of this is basically what's seen here minus the references to '@rollup/plugin-typescript'.

rollup.config.mjs

(Either we make the package-generator preset have type: "module" and give the transpiled output a .cjs extension, or we keep it a CommonJS package and use an .mjs extension for any ESM code.)

import commonjs from '@rollup/plugin-commonjs';
import resolve from '@rollup/plugin-node-resolve';
import json from '@rollup/plugin-json';
import typescript from '@rollup/plugin-typescript';

export default {
  input: 'src/index.ts',
  output: {
    file: 'lib/index.js',
    format: 'cjs',
    exports: 'auto',
    interop: 'auto',
    sourcemap: true
  },
  plugins: [
    resolve({
      extensions: ['.js', '.ts', '.json'],
      // Look in node_modules for dependencies
      preferBuiltins: true,
      // Whether to use the "main" or "module" field in package.json
      mainFields: ['main', 'module']
    }),
    commonjs({
      // Include all files in node_modules
      include: /node_modules/,
      // Enable transformations of ES modules in node_modules
      transformMixedEsModules: true,
      // Handle requiring JSON files
      ignoreDynamicRequires: false
    }),
    typescript({
      tsconfig: './tsconfig.json',
      sourceMap: true
    }),
    json()
  ],
  // Mark certain packages as external; this tells Rollup not to try to
  // transpile this package's code. You may opt into this for any dependency
  // that exports a CommonJS version.
  external: [
    'atom'
  ]
}

tsconfig.json

{
  "compilerOptions": {
    "target": "es2018",
    "module": "esnext",
    "lib": ["es2018", "dom"],
    "declaration": true,
    "outDir": "./lib",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "resolveJsonModule": true,
    "sourceMap": true,
    "moduleResolution": "bundler"
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "lib"]
}

What benefits does this feature provide?

A friendlier alternative to the status quo.

It used to be that Electron didn't understand modern JS syntax at all, so the best option was to introduce built-in Babel transpiling on a file-by-file basis. That makes less sense now that all the fancy stuff is supported except for ESM import. And it doesn't do anything about dependencies, which limits its usefulness.

If we made this the package-generator preset for a package, users wouldn't have to stress out about whether their NPM dependencies are ESM or CommonJS.

---

As for the TypeScript thing: I think we should meet people where they are. Doesn't mean we have to teach them to write community packages in TypeScript, but we should show them what the TypeScript route would look like, and ensure they enjoy the niceties inherent to that route (like type inference on builtin Atom APIs via the @types/atom package).

Any alternatives?

The alternatives are already implemented: per-file transpilation within Pulsar. This is not very useful these days for JS and even less useful for TypeScript — which can theoretically be transpiled by Pulsar, though the dependency that does that work is many years old and several versions behind.

Better, I think, to move away from that model as much as we can, even if it does introduce a transpiler step.

Other examples:

No response

[savetheclocktower]

This might be more salient — the ESM-to-CommonJS transpiling, at least — once we're on PulsarNext. Someone writing a package for current Pulsar is bound to pull in dependencies that still support Node 14, and those are less likely to have gone ESM-only.

But Electron 30 is on Node 20.11.1, and there are a whole bunch of packages that are ESM-only and target Node 20 or greater.

[savetheclocktower]

I made pulsar-hover using the TypeScript preset described above and it's gone quite well.

There's one way that it didn't go well, but it's a bit of an edge case: for a while I thought I'd need to bring in something React-like, and I was trying very hard to make it something instead of React, like SolidJS. But SolidJS has very opinionated choices about how JSX should be configured in a TypeScript toolchain, and I wasn't able to make all the pieces of the chain happy. I forget the exact error messages, but I'm logging it here so I don't forget it.