added script to upload vanilla .svelte components.

[BlackFenix2]

I added a script to preprocess svelte files with different Template,Script and Style languages and spit out vanilla .svelte files. This is to serve 2 use cases for a component library.

• Sapper-friendly compiled .svelte components
• Allows use by consumers that do not have preprocessors in their current project

This may help Sapper projects more than anything else, bare bones svelte projects shouldn't have a problem loading modules.

[firefish5000]

Failing to see this, I wrote one as well. POC is available via npx @tdi/svcli --preprocess. Note though the project is a badly and blindly written scaffold cli for my (eventual?) personal use that should not be relied on. Rest of this post is in regards to this PR & script.

As far as non-svelte files are concerned. As much as I would like better integration, I believe typescript, postcss, and etc. can use their own compilers as per norm without any issue. We could simply tell users to add lines to package.json like

package.json

...
"scripts": {
  "preprocess": "npm run preprocess:svelte; npm run preprocess:typescript",
  "preprocess:svelte": "node script.js",
  "preprocess:typescript": "tsc --declaration --outDir wherever/preprocessed/files/go"
}
...

and add calls to all preprocessors in package.json via (possibly utilizing npm-run-all). If you do want an all in one solution, this may be the tipping point at which it becomes worth it to write a plugin for rollup/webpack to dump the processed files.

As for improvements, the only major thing I can think of is using an arrays instead of just autoPreprocess() and .svelte extension so it is more obvious to users how to port their rollup/webpack config should they use other preprocessors (like mdsvex). And putting the arrays somewhere accessible, like the top of the script, or in its own config file (so rollup and webpack can import it and everything stays in sync).

svelte-preprocess.config.js

const sveltePreprocess = require("svelte-preprocess")
const {mdsvex} = require("mdsvex")
const generatePreprocess = ({sourceMap}={})=>[
    sveltePreprocess({sourceMap: sourceMap ?? false}),
    mdsvex(),
  ]
module.exports={
  generatePreprocess: generatePreprocess,
  preprocess: generatePreprocess(),
  extensions: [".svelte",".svx"],
  outDir: "./preprocessed",
  srcDir: "./src",
}

You can then use them in rollup, webpack, and the script. For the script, I would do

script.js (or a more descriptive filename)

   const {extensions,preprocessors} =  require('./svelte-preprocess.config.js')
   ...
   // process .svelte file
   // Use endsWith instead of includes so we don't process .svelte.disabled, .svelte.css and other improper extensions.
   if (extensions.some(ext=>file.endsWith(ext))) {
     // run autopreprocessor
     await parseSvelte(sourceFile, distFile);
   }
   ...
   svelte.preprocess(source, preprocessors, {
      filename: path.basename(destination),
    })

In my version, I also ~~gave the ability to~~ required setting the source directory and output directory. Not sure I would call that an improvement or addition of entirely pointless config options.

Dependencies

IMHO, There is no need real need for component-template to ship with svelte-preprocess, typescript, or any other preprocessors added to devDependencies. However, svelte doesn't offers an easy way to output the preprocessed files. So we should at least ship with this script, directions on how to use it, and a list of anything extra that must be installed to run it (should be none, as svelte-preprocessor is just one svelte processor implementation, not the all-in-one solution).

Actually, might be better if this script lived in its own package and we just had "preprocess": "npx preprocess-svelte" in package.json script, leaving installation entirely optional.

Also include some (ideally short) directions on how to enable compiling a few other common languages. I gave an example for typescript using tsc. If that is method is acceptable, just a short explanation of what it does should suffice.

[bluwy]

I also did something similar for my library. with the extra step of removing the lang attribute too since I saw that svelte-preprocess actually checks the attribute before preprocessing it. Just to make sure, is this step necessary too?

[dummdidumm]

I also did something similar for my library. with the extra step of removing the lang attribute too since I saw that svelte-preprocess actually checks the attribute before preprocessing it. Just to make sure, is this step necessary too?

That's a good catch. If the preprocessed output still contains the lang attribute it would be preprocessed again.

[firefish5000]

I also did something similar for my library. with the extra step of removing the lang attribute too since I saw that svelte-preprocess actually checks the attribute before preprocessing it. Just to make sure, is this step necessary too?

That's a good catch. If the preprocessed output still contains the lang attribute it would be preprocessed again.

Would this also happen if we enable defaults in svelte-preprocess? I prefer to specify lang rather than using defaults as well, but I am curious if the preprocessor gets triggered when a user set their defaults to pug, ts, and sass and then installs pure svelte components.

[dummdidumm]

I guess it will. Maybe the default settings hurt more overall than they help..

[firefish5000]

I guess it will. Maybe the default settings hurt more overall than they help..

I wouldn't say they are the issue. When I enable a preprocessor, I expect it to run on my code that I wrote in my current project. Touching code anywhere else is a taboo which I should need to explicitly enable. The issue is, atm, it seems that taboo is the default behavior, non-optional, and for some people it is even something that got forced down their throats because they wanted to install some component that used a preprocessor.

While this script is certainly a necessity that will finally stop adding converts to our cult and give the devs a chance their throats with mouthwash. Perhaps we still need a bit more to change with the ecosystem before it will become possible to achieve a full and proper solution to our problem. This conversation leaves me wondering

1. Is it currently possible to limit preprocessing to certain paths (not just in the cli, but in our bundler as well)? a. If not, is there a reason not to? b. Are there other ways to handle this? What do other languages do?
2. Is it likely that running preprocessors on pure svelte files will cause issues in practice (and are we already doing that)?

For the first question, I know svelte.preprocess gives at least a filename to the preprocessors. If it gave the full path, then there would be a way to resolve this delima in the individual preprocessors. As for methods of resolving this issue... to the best of my limited knowledge, most preprocessed languages offer options akin to src/out file/dir and include/exclude globs. The only other option I know of is to use a different file extension... And now I am thinking that was the intent of giving us a filename. pngwn may have thought this through even better than the official preprocessor, as mdsvex only processes .svx extension by default.

Now, does re-running any of the preprocessors seem likely to cause issues in practice (other than wasted cycles)? I would imagine so, though it depends on what preprocessor and what plugins are enabled by the end dev. For instance:

1. Postcss with the sugarss parser will fail miserably at parsing any valid css, as sugarss does not permit brackets nor semicolons.
2. I can't imagine pug working at all unless the preprocessor is smart enough to do nothing when pug syntax isn't used (but as a dev I would expect an error to be thrown).

Typescript might be fine, since (iirc) type checking isn't done by svelte-preprocess, and js is valid ts under normal circumstances.

Both of those cases should be possible to test now using either this script, or by enabling defaults in svelte-preprocess and installing any pure-svelte component. I have not tried either of them yet, so let me know if you do.

TL;DR We have an unresolved issue in regards to svelte-preprocess that will likely cause issues even with this script. The issue being it can run on pure svelte files that are outside the project we are in. The solution needs to work when svelte.preprocess is called by both this script and rollup/webpack. To fix this, we could modify svelte.preprocess to enable a include/exclude: Array<Glob> or similar config field to limit preprocessing to specific files. With an exclude field that blocks at least node_modules by default. Or we could make svelte-preprocess only process specific extension so it doesn't run on pure svelte files. This was probably the intent of whoever wrote svelte.preprocess if the filename is the only extra context it gives you.

[dummdidumm]

cc @kaisermann who is the author of svelte-preprocess.

The fact that svelte-preprocess does run on normal svelte files is the desired behavior. It does check the lang/type attribute to know what to preprocess in which way. defaultLanguages makes this a problem because then it's always run. Making svelte-preprocess run only on non-svelte files is a no-go for me. Some kind of include/exclude seems good.

[firefish5000]

Making svelte-preprocess run only on non-svelte files is a no-go for me. Some kind of include/exclude seems good.

Sounds good to me. New question, does the script need to standardize extensions to .svelte? Taking mdsvex's .svx back into account, this script (and mine) currently just changes the directory and keeps the filename. But I believe a standard svelte installation will only look at .svelte files. If it should mangle the extensions, easiest option I see is to tack on .svelte, and throw on conflict (like src/Component.svx and src/Component.svx.svelte both exists in src).

[kaisermann]

Those exclude/include props would only function as filters, right? The exclude option is very straight-forward to grasp: makes svelte.preprocess skip preprocessing a filename that matches one of the globs. What would include mean?

[dummdidumm]

include would mean the opposite: only preprocess files that match the glob. Starting with exclude would probably be enough.

[firefish5000]

Starting with exclude would probably be enough.

Agreed, include is just a convenient way to exclude everything else in our case. Totally unnecessary, could even be more confusing to have it since improper usage would easily conflict with svelte's extensions option.