rollup Why does the `preserveModules` option generate `node

Documentation Is:

[ ] Missing
[ ] Needed
[ ] Confusing
[x] Not Sure?

As described here in the docs, I'm using the plugin-node-resolve plugin to include the dependencies of my library in the final bundle.

I'm also using the external option as described here to exclude the peerDependencies. So the final rollup config looks something like this:

import path from 'path';
import resolve from '@rollup/plugin-node-resolve';
import babel from '@rollup/plugin-babel';

export default {
  input: 'src/index.js',
  output: {
    dir: './dist',
    format: 'es',
    preserveModules: true,
  },
  external: [
    'prop-types',
    'react',
    'react-dom',
  ],
  plugins: [
    resolve({
      extensions: ['.js', '.jsx', '.json'],
    }),
    babel({
      babelHelpers: 'bundled',
      exclude: 'node_modules/**',
    }),
  ],
};

The problem is that after the build, the dependencies of the package are generated in this location: dist/node_modules/PACKAGE_NAME. And their import statements in other files are something like this: import SOMETHING from '../../../node_modules/PACKAGE_NAME/index.js';

And on the other hand, when I build the package on the consumer project (using prepare script), this nested node_modules directory doesn't exist (possibly because npm and yarn flatten the node_modules structure).

So I was wondering is there any option to fix this problem? e.g. a way to change the name of these generated node_modules folders and their import statements to something else...

Jul 21 '20 12:07 ghbakhtiari

The folder is created because preserveModules is preserving the original file names and directory structure. It does not understand that node_modules is something special, though. Also, all exports are rewritten in a way that does not depend on Node's dependency resolution algorithm.

If you want to change names, you might have some luck using the entryFileNames option with a function. I am not 100% sure we implemented it for preserveModules, would be interesting if you gave it a go: https://rollupjs.org/guide/en/#outputentryfilenames

Jul 21 '20 15:07 lukastaegert

@lukastaegert Thanks for the reply.

I just tried entryFileNames both with and without the preserveModules option, and unfortunately got this error (both times):

[!] TypeError: name.replace is not a function

In the error's stack trace it seems that the entryFileNames is passed as pattern and finally produces an error in this function:

export function sanitizeFileName(name: string): string {
  return name.replace(/[\0?*]/g, '_');
}

It's a bit strange because the docs just says that this option supports both string and function 🤔 Am I doing something wrong?!

Jul 22 '20 08:07 ghbakhtiari

Did the function return a string? 😉

Jul 22 '20 09:07 lukastaegert

And are you on latest Rollup?

Jul 22 '20 09:07 lukastaegert

Yes it's returning some random string 😀 The important problem is that I put some logs inside the function and it's not called at all.

The versions are:

    "@rollup/plugin-alias": "^3.1.1",
    "@rollup/plugin-babel": "^5.0.4",
    "@rollup/plugin-commonjs": "^13.0.0",
    "@rollup/plugin-node-resolve": "^8.1.0",
    "@rollup/plugin-replace": "^2.3.3",
    "rollup": "^2.18.1",
    "rollup-plugin-terser": "^6.1.0"

Jul 22 '20 09:07 ghbakhtiari

As a first step I would update Rollup as that feature is a little newer than your minimum Rollup version: https://github.com/rollup/rollup/releases/tag/v2.20.0

Jul 22 '20 09:07 lukastaegert

The update fixed the function problem 👍

But now when I try to rebuild the full path (to replace the node_modules with something else) with something like this:

    entryFileNames: ({ facadeModuleId }) => {
      if (!facadeModuleId.includes('node_modules')) {
        return '[name].js';
      }

      return facadeModuleId.replace('node_modules', 'dependencies');
    },

I get this error:

[!] Error: Invalid pattern "/SOME_FOLDERS/dependencies/SOME_PACKAGE/index.js" for "output.entryFileNames",
patterns can be neither absolute nor relative paths and must not contain invalid characters.

It seems that I can only nest the result in some sub_directories but can't change the path that the file already has.

For example if I write it like this:

    entryFileNames: ({ facadeModuleId }) => {
      if (!facadeModuleId.includes('node_modules')) {
        return '[name].js';
      }

      return 'dependencies/[name].js';
    },

then it will nest all of the node_modules/SOME_PACKAGE/... files in node_modules/SOME_PACKAGE/dependencies/....

Jul 22 '20 10:07 ghbakhtiari

Ok, this should probably be improved. I would have thought that [name] contains the full path, but apparently it is prepended. Not ideal. For now, you can manually convert the facadeModuleId to a valid path fragment that does not start with a . or /, i.e. /some/path/node_modules/lib/file.js -> dependencies/lib/file.js. Node's path.relative can be helpful here.

Jul 22 '20 11:07 lukastaegert

Ah, that will still not solve your problem, my bad

Jul 22 '20 11:07 lukastaegert

Exactly, the problem is that whatever we return from this function, it would only be appended to the end of the path of the file.

Jul 22 '20 12:07 ghbakhtiari

I fear we cannot fix this without a breaking change at the moment

Jul 22 '20 12:07 lukastaegert

Well, currently the isPlainPathFragment function returns false when the name starts with "/", "./" or "../", and then throws an error in renderNamePattern.

So instead of throwing, if we just add this functionality where these types of names would change the path as well, would it still be a breaking change? I mean in absolute cases (starting with "/"), the path would be completely replaced. And in the relative cases (starting with "./" or "../") a new path would be resolved from them...

Also, another way is to add a new option besides entryFileNames which supports this functionality (change the path as well...)

Jul 22 '20 14:07 ghbakhtiari

I'm encountering this issue, although it's slightly different in that I'd also like to have control over the modules that get output from preserveModules: true. Right now, my src/ directory path is preserved, so I end up with dist/src/index.js when I really only want dist/index.js.

I think the underlying issue here is with @rollup/plugin-node-resolve. See https://github.com/rollup/plugins/issues/321 and https://github.com/rollup/plugins/issues/277, both of which have been closed due to inactivity.

I can try reproducing soon, but right now my work-around has been to use rollup-plugin-rename to restructure the output, and also to handle the relative imports that otherwise would have problems due to this restructuring.

Sep 15 '20 06:09 davidroeca

I was having the same issue and since I didn't know if it was a rollup bug or not I just created a plugin to fix this called rollup-plugin-rename-node-modules. I'm using it in my project and now it works.

Oct 02 '20 22:10 Lazyuki

I'm encountered same issue.

preserveModules: false gives this

dist/
├── index.js
├── second.js

preserveModules: true gives this

dist/
├── src/
│   ├── index.js
│   ├── second.js
├── node_modules/

To workaround it, I combined davidroeca's rollup-plugin-rename and Lazyuki's node_module renaming to external. The final solution:

import commonjs from '@rollup/plugin-commonjs';
import resolve from '@rollup/plugin-node-resolve';
import babel from 'rollup-plugin-babel';
import rename from 'rollup-plugin-rename';

export default {
    input: 'src/index.js',
    output: {
        dir: 'dist/',
        format: 'cjs',
        preserveModules: true,
    },
    plugins: [
        commonjs(),
        resolve({
            resolveOnly: ['lodash-es'],
            // modulesOnly: true,
        }),
        babel({
            runtimeHelpers: true,
        }),
        rename({
            include: ['**/*.js'],
            map: (name) => name
                .replace('src/', '')
                .replace('node_modules/', 'external/')
                .replace('../../external', '../external'),
        })
    ],
};

And the output will look like this:

dist/
├── index.js
├── second.js
├── external/

Oct 18 '20 17:10 shrpne

@shrpne Thank you so much, this was causing me so much pain trying to solve!!

It's really worrying that this is the fix, assuming this is just a temporary workaround as this must be a bug that needs fixing?

For me the two issues are that it creates an src folder and after fixing that, it's unable to properly find the node_modules folder

Nov 25 '20 12:11 Quirksmode

Any progress on this?

Sep 02 '21 21:09 tbaustin

I'm experiencing this issue also. Adding this to my config has worked pretty well for me in the meantime:

const pkg = require('./package.json');

const externalPackages = [
  ...Object.keys(pkg.dependencies || {}),
  ...Object.keys(pkg.peerDependencies || {}),
];
// Creating regexes of the packages to make sure subpaths of the
// packages are also treated as external
const regexesOfPackages = externalPackages
  .map(packageName => new RegExp(`^${packageName}(\/.*)?`));

export default {
  external: regexesOfPackages,
  // ...
};

Sep 24 '21 11:09 kirkobyte

I am using the following code to solve my issue

function entryFileNames(chunkInfo) {

	var matches = /(?:\bsrc[\\|\/])(?<path>.*?[^\\\/]+)(?:\.svelte)$/g.exec(chunkInfo.facadeModuleId);
	if( matches != null && matches.length > 1 ){
		return matches[1] + ".js";
	}

	return "[name].js";
}

{
		input: ['src/Widget1.svelte', 'src/Widget2.svelte', ...],
		output: {
			sourcemap: true,
			format: 'amd',
			name: 'app',
			dir: 'public/build/',
			entryFileNames: entryFileNames
		}
}