[feat]: dev server plugin

[bennypowers]

see https://github.com/open-wc/create/issues/48#issuecomment-900306742

[hsablonniere]

Hey, I'm also interesed in such a dev server plugin :wink:

We managed to make something but it clearly needs some love.

• The plugin:
  • https://github.com/CleverCloud/clever-components/blob/master/web-dev-server.config.js#L102-L109
• The generateCustomElementsManifest implementation:
  • https://github.com/CleverCloud/clever-components/blob/master/tasks/cem-analyzer.js

Here are some "limitations" we encountered:

• There's no easy way to get the raw JSON out of a JS module call or a CLI call
  • That's why we decided to write to a temporary file with a CLI call and then read from this file
• There's no easy way to override the outdir from the CLI
  • Something like --outdir would be nice but would it be a good idea to make it override what's in the config file?
  • That's why we went for an environment variable that we set before the CLI call and that the config file uses
• The CLI call always updates the customElements field of the package.json
  • That's why we read the package.json before and write (rollback) the contents afterwards

:tada: Now that we have this plugin, we have very performant HMR when we edit our components and if we edit the docs, we can manually refresh the browser to see the updates of CEM in Storybook.

Of course, it would be even better with a cleaner and official implementation.

• Do you already know how you would implement such a plugin?
• Do you want/need some help?

[thepassle]

There's no easy way to get the raw JSON out of a JS module call or a CLI call

Do you mean analyze on a per-module basis?

import { create } from '@custom-element-manifest/analyzer/browser/index.js';
const manifest = create({ modules });

will give you the json for a module, but if its only 1 module of your project, then the info in the output CEM will be very incomplete, because it needs the other modules in your project.

Additionally, using the --dev flag will output the CEM as JSON to stdout as well

There's no easy way to override the outdir from the CLI

There already is an --outdir option

The CLI call always updates the customElements field of the package.json

I thought we already made this configurable, but yeah we can add this. It should default to true, and then can be overwritten to be false

[hsablonniere]

Do you mean analyze on a per-module basis?

Sorry, I meant a JS module I could import and call to generate the CEM. I wasn't referring to the module (or modules) I want to analyze.

Additionally, using the --dev flag will output the CEM as JSON to stdout as well

Yes, I tried that when you mentioned it in Slack but extracting the JSON from this is really tricky.

There already is an --outdir option

:scream: Oh, I'm really sorry, I totally missed this one. It does not work with absolute paths so it's a bit tricky to use with /tmp. :bulb: But I just thought about using some node_modules/.cem-tmp or something. It will be way easier.

I thought we already made this configurable, but yeah we can add this. It should default to true, and then can be overwritten to be false

Fair enough.

Again, if you want/need some help on this, we're here :hugs:

[thepassle]

Sorry, I meant a JS module I could import and call to generate the CEM. I wasn't referring to the module (or modules) I want to analyze.

Im not really sure what you mean with this point. You can import the create function and generate a CEM

[hsablonniere]

Im not really sure what you mean with this point. You can import the create function and generate a CEM

I would need to import all my modules, load the config file with my CEM analyze plugins, and write a lot of boilerplate to generate the same custom-elements.json output as the CLI does, right?

[thepassle]

I would need to import all my modules, load the config file with my CEM analyze plugins, and write a lot of boilerplate to generate the same custom-elements.json output as the CLI does, right?

You would need to read the modules you want to analyze, and pass any custom plugins you're using to create. Not sure what kind of boilerplate you're expecting to need to write?

I feel like I'm misunderstanding your point, could you explain your original point in a different way perhaps? <aybe with an example?

There's no easy way to get the raw JSON out of a JS module call or a CLI call

[hsablonniere]

Right now we have custom WDS plugin which does this:

async serve (context) {
  if (context.path === '/dist/custom-elements.json') {
    return generateCustomElementsManifest();
  }
},

The custom-elements.json output contains the documentation for all our components and is used by our Storybook to display the props table, configure controls and other details...

In generateCustomElementsManifest(), I expect the same output as when I generate the custom-elements.json with the CLI cem analyze --litelement, before publishing to npm.

This means, I expect this function to rely on what I already configure in custom-elements-manifest.config.mjs:

• the list of plugins to load and use
• the list of JS module (components) to load and analyze with glob

From what I understood in this documention page, I can achieve the same thing with this browser API:

import { create } from '@custom-element-manifest/analyzer/browser/index.js';

When I say boilerplate, I mean:

1. I'll have to explicitly load the same list of plugins I configured in custom-elements-manifest.config.mjs
   • I guess I could load the config file and use the default export and the plugins property :+1:
2. I'll have to manually load the same list the modules I configured in custom-elements-manifest.config.mjs
   • I could use the config file globs property but then, I'll have to load the modules that match this and then load their content etc...

The implementation of generateCustomElementsManifest() would look like this:

// I wrote this directly in the issue without testing but you get the idea

import cemConfig from './custom-elements-manifest.config.mjs';
import { ts, create, litPlugin } from '@custom-element-manifest/analyzer/browser/index.js';
import glob from 'some-glob-lib';
import fs from 'fs/promises';

export async function generateCustomElementsManifest() {

  const pathList = await glob(cemConfig.globs);

  const fileList = await Promise.all(pathList.map((filepath) => {
    const contents = await fs.readFile(f, 'utf8');
    return { filepath, contents };
  }));

  const modules = fileList.map((file) => {
    return ts.createSourceFile(
      file.filepath,
      file.contents,
      ts.ScriptTarget.ES2015,
      true,
    );
  });

  const manifest = create({
    modules,
    plugins: cemConfig.plugins,
    dev: cemConfig.dev
  });

  return manifest;
}

This is the "boilerplate" I'm talking about, I feel like this probably already exist somewhere in the CEM analyzer codebase since it's handled by the CLI. That's why I went for a let's just get the output from the CLI (for now) so we're sure we get the same thing as we'll have in production.

Let me know if this example helps or not :wink:

[pdesoyres-cc]

With @hsablonniere, we managed to implement it like that:

import { cli } from '@custom-elements-manifest/analyzer/cli.js';

const path = '/dist/custom-elements.json';

// This plugin generates and serves the CEM on demand.
export const cemAnalyzerPlugin = {
  name: 'cem-analyzer-plugin',
  // avoid @rollup/plugin-json plugin from caching the result to enforce CEM generation for every call
  transform (context) {
    if (context.path === path) {
      return {
        transformCache: false,
      };
    }
  },
  async serve (context) {
    if (context.path === path) {
      const cemJson = await cli({
        argv: ['analyze'],
        noWrite: true,
      });

      return JSON.stringify(cemJson);
    }
  },
};

• we can get the JSON without writing a temporary file
• the package.json is not modified during the process (this is now the default behavior :+1: )

:pray: Thank you to all the contributors for this new implementation of the cli.js. :bulb: It would be even better if we did not need to play with argv

[hsablonniere]

@thepassle @bennypowers Are you interested in moving forward with this. We could contribute the plugin to open-wc :wink: