Support embedded JavaScript/TypeScript

[remcohaszing]

There are several frameworks / languages in JavaScript land that contain embedded JavaScript or TypeScript. Some languages that come to mind are:

• Astro (Volar)
• Ember (Volar)
• HTML script tags (Not yet imlemented)
• Markdown code blocks (Not yet imlemented)
• MDX (Volar)
• Svelte (Custom)
• Vue (Volar)

I am the maintainer of the MDX language tooling, so I can best sketch the situation from that point of view. MDX is basically markdown with JSX tags. Just like TypeScript or JSX, it gets compiled to JavaScript.

Currently Volar offers a solution for this. To support this, Volar has an API to abstract away creating a TypeScript plugin. This works ok, but TypeScript plugins depend on monkey-patching TypeScript provided objects. If the TypeScript team gets more picky on the public interface, that might break all of these integrations.

I think maybe this rewrite to Go would be a good opportunity to also think of an interface to add builtin support for such languages.

Volar plugins work by converting text content to a virtual file with mapped JavaScript or TypeScript content. For example, this:

export function Local() {}

<div local={<Local />} injected={<Injected />} string="string" boolean />

<div local={<Local>{null}</Local>} injected={<Injected>{null}</Injected>} string="string" boolean>
  paragraph
</div>

becomes this, with some mappings:

/* @jsxRuntime automatic
@jsxImportSource react */
export function Local() {}


/**
 * @internal
 *   **Do not use.** This function is generated by MDX for internal use.
 *
 * @param {{readonly [K in keyof MDXContentProps]: MDXContentProps[K]}} props
 *   The [props](https://mdxjs.com/docs/using-mdx/#props) that have been passed to the MDX component.
 */
function _createMdxContent(props) {
  /**
   * @internal
   *   **Do not use.** This variable is generated by MDX for internal use.
   */
  const _components = {
    // @ts-ignore
    .../** @type {0 extends 1 & MDXProvidedComponents ? {} : MDXProvidedComponents} */ ({}),
    ...props.components,
    /** The [props](https://mdxjs.com/docs/using-mdx/#props) that have been passed to the MDX component. */
    props,
    /** {@link Local} */
    Local
  }
  _components
  return <>
    <div local={<Local />} injected={<_components.Injected />} string="string" boolean />
    <div local={<Local>{null}</Local>} injected={<_components.Injected>{null}</_components.Injected>} string="string" boolean>
    <>
    {''}
    </>
    </div>
  </>
}

/**
 * Render the MDX contents.
 *
 * @param {{readonly [K in keyof MDXContentProps]: MDXContentProps[K]}} props
 *   The [props](https://mdxjs.com/docs/using-mdx/#props) that have been passed to the MDX component.
 */
export default function MDXContent(props) {
  return <_createMdxContent {...props} />
}

// @ts-ignore
/** @typedef {(void extends Props ? {} : Props) & {components?: {}}} MDXContentProps */

This is not actually what compiled MDX looks like. It’s similar, but it contains injected helper code to help / trick TypeScript.

I think TypeScript could provide a similar API. Some pseudo code:

import { Plugin } from 'tsgo'

const plugin: Plugin = (typescript) => {
  return (project) => {
    project.registerFileType({
      extension: ['.mdx'],  // For the compiler
      languageID: ['mdx'],  // For the language server
      scriptKind: typescript.ScriptKind.JSX,  // describes the virtual file content
      parse(content) {
        // Turn the MDX string content into something TypeScript understands
      }
    })
  }
}

export default plugin

There are still many open ends here:

• Should the CLI support plugins? There’s definitely user demand.
• Should emitting be supported? This is useful for some, but not all languages. Also the content that provides the best TypeScript experience often doesn’t match the compile output exactly.
• Should emitting type definitions be supported? This is useful for some, but not all languages.
• What should the source content be parsed into? A string, like Volar? Or a TypeScript AST?
• Nodes from the virtual code might not exist in the source file.
• What languages should be supported? Currently it’s all JavaScript, but MDX also has a Rust API which might be useful to speed things up.

[NullVoxPopuli]

emitting type definitions would be hugely useful -- not just for library authoring (for ember's .gts files (or .svelte, .vue, etc)), but also for having a way to interact with typedoc and other tsc-using tools