rfc: add Webpack loader to minify CSS and HTML templates

[solanors20]

💬 RFC

Right now FAST uses rollup-plugin-transform-tagged-template to minify HTML and CSS templates from Web Components. There are projects that can't use this plugin because they use Webpack instead of Rollup. It would be beneficial to have a webpack loader that can do a pre-processing transformation like the roll-up plugin does, to reduce the size of the bundles that use FAST.

🔦 Context

Problem Minimizers used in Webpack like Terser doesn't transform tagged templated strings causing bigger bundles size for production. If you search on the internet solutions for this problem for Webpack there are almost no results.

Related work There are a couple of loaders/plugins that try to solve this, but sometimes they only support html (like this one) or is for a specific library (like this one)

Ideas The rollup-plugin-transform-tagged-template uses Babel plugins to parse JavaScript to an AST (Abstract Syntax Tree) and searches for tags (in this case we want CSS and HTML tags), then it applies transformations to the string given by the user. Fluent-ui is using this transformations to minify the templates: https://github.com/microsoft/fluentui/blob/ecb0e9b12665a05353f64f1b69981584c3addbc0/packages/web-components/build/transform-fragments.js

There are other parsers that we could be used like Acorn that can be used to traverse the AST searching for tags.

💻 Examples

This is a Webpack loader example based on rollup-plugin-transform-tagged-template . It has the same limitation that it will require specifics transformations given by the user.

const _parser = require("@babel/parser");

const _traverse = require("@babel/traverse");

const _generator = require("@babel/generator");

const _loaderUtils = require("loader-utils");

module.exports = function (source) {
    const loaderOptions = _loaderUtils.getOptions(this);
    const options = {
        parserOptions: loaderOptions.parserOptions,
        tagsToProcess: loaderOptions.tagsToProcess,
        transformer: loaderOptions.transformer
    };
    const ast = (0, _parser.parse)(source, options.parserOptions);
    (0, _traverse.default)(ast, {
        TaggedTemplateExpression(path) {
            if (options.tagsToProcess.includes(path.node.tag.name)) {
                for (const quasi of path.node.quasi.quasis) {
                    const transformedData = options.transformer(quasi.value.raw);

                    quasi.value.raw = transformedData;
                    quasi.value.cooked = transformedData;
                }
            }
        }
    });
    const result = (0, _generator.default)(ast, { compact: true }, source);
    return result.code;
};

[EisenbergEffect]

Can we do this in a way that we have a core shared library that is used by both rollup and webpack? /cc @radium-v who has done some work on the Rollup side of this.

[radium-v]

@EisenbergEffect yes, and I have a solution I've been experimenting with that does exactly that.

[solanors20]

I think most of the code could be shared by a library. The only things that probably can't be shared would be specific to the webpack and rollup frameworks (options passed to the plugin, export of the module, schema validation, etc).

[RehanSaeed]

@EisenbergEffect @radium-v Would you be interested in a PR for the above webpack loader?

[EisenbergEffect]

I think we'd definitely be interested. Maybe you can get with @radium-v (jak) on Discord to work out a quick plan of action so work isn't duplicated or wasted. Then maybe update this thread with a summary of that.

[nicholasrice]

Another lib I found for this is https://github.com/asyncLiz/minify-html-literals. It is the backbone of https://www.npmjs.com/package/rollup-plugin-minify-html-literals?activeTab=readme, but I haven't been able to track down a webpack loader that implements it.

[rajsite]

We are using rollup-plugin-transform-tagged-template and copied in the FAST transform-fragment into our build but are finding that the plugin does not seem to be sourcemap aware and it was breaking debugging via sourcemap (line numbers, etc for breakpoints were wrong). Haven't found a workaround yet and are temporarily removing the plugin.

[solanors20]

I ended up using a simple replace with regex instead of the AST implementation because it was conflicting with other minifier (terser). But as @rajsite said, this can break the debugging. I haven't found a solution for that yet.

const _loaderUtils = require("loader-utils");

module.exports = function (source) {
    const loaderOptions = _loaderUtils.getOptions(this);
    const options = {
        parserOptions: loaderOptions.parserOptions,
        tagsToProcess: loaderOptions.tagsToProcess,
        transformer: loaderOptions.transformer
    };

    options.tagsToProcess.forEach(tag => {
        var regex = "[\\s+]?[\\s+=]?" + tag + "\\s*`{1}([^`]*)`{1}";
        var regexExp = new RegExp(regex,"g");
        source = source.replace(regexExp, function(a, b){ 
            const tr = options.transformer(b);
            return tag + '\`' + tr + '\`';
        });
    });
    
    return source;
};