Multiple globally installed presets and plugins don’t work

[pvdlg]

Following the example in the README.md.

// example.md
* Hello

[World][]

// .remarkrc
{
  "plugins": [
   "remark-preset-lint-recommended",
   ["remark-lint-list-item-indent", false]
  ]
}

$ remark . 

example.md
       1:3  warning  Incorrect list-item indent: add 2 spaces  list-item-indent         remark-lint
  3:1-3:10  warning  Found reference to undefined definition   no-undefined-references  remark-lint

The list-item-indent rule should have been disabled, but it's still active.

[wooorm]

Could you rub with DEBUG="*" remark . and post the results?

This problem can occur if remark-preset-lint-recommended loads a different remark-lint-list-item-indent (such as if versions mismatch).

[pvdlg]

I create a gist with the logs and the .remarkrc: https://gist.github.com/pvdlg/b1eca9123b975ee66dd95cbde1aa134c

In this example I was trying to disable remark-lint-maximum-line-length.

This problem can occur if remark-preset-lint-recommended loads a different remark-lint-list-item-indent (such as if versions mismatch).

I think this situation can happen very frequently as npm and yarn tries to flatten the dependency tree.

So when remark-preset-lint-recommended specifies a dependency to remark-lint-list-item-indent it's entirely possible that there is something else in the dependency tree that also depends on remark-lint-list-item-indent and you might end up with this situation.

Wouldn't it be better to depend on module/rule name rather than Object identity?

[wooorm]

The configuration you posted in the gist in different from what you posted above. Which one is this issue about?

Red flattening: that's what enables duping on object equality, otherwise it wouldn't be possible at all!

[wooorm]

Where is your configuration file compared to the readme file you are processing? The logs show that it isn't found? What are you using to run remark?

[pvdlg]

I deleted the reproduction env I created yesterday and I posted a similar case instead. Sorry for the confusion.

Here is the gist with the case initially mentioned: https://gist.github.com/pvdlg/1ad2570fa203a1270f1b9e66d1930bff

The file structure is:

.
+-- test_remark/
|   +-- .remarkrc
|   +-- example.md

The logs show that it isn't found? What are you using to run remark?

I use the CLI remark . which is installed with npm -g

[wooorm]

Hmm, how weird! What are the versions of node, npm, remark-lint, the preset, and the rule? Can you check if the rule your turning off is deduped? Is it there one? Or is there one in the preset as well? Does rm -rf node_modules fix anything?

[pvdlg]

The new gist I posted also has the error unified-engine:find-up No files found.

Even thoughremark finds 2 warnings for list-item-indent and no-undefined-references so I guess the configuration is read...

[wooorm]

Yeah, the configuration is read, I misread the logs as saying otherwise as well, but a later line shows its reading the configuration file anyway!

[pvdlg]

• npm: 5.5.1
• node: v8.9.1

$ npm ls -g remark-lint                                                                                                                       23ms 
/Users/vanduynslager/.npm-packages/lib
├── [email protected] 
├─┬ [email protected]
│ └── [email protected] 
├─┬ [email protected]
│ └── [email protected] 
└─┬ [email protected]
  └── [email protected] 

$ npm ls -g remark                                                                                                                        8s 672ms 
/Users/vanduynslager/.npm-packages/lib
├── [email protected] 
├─┬ [email protected]
│ └── [email protected] 
├─┬ [email protected]
│ └── [email protected] 
└─┬ [email protected]
  └─┬ [email protected]
    └── [email protected] 

$ npm ls -g remark-preset-lint-recommended                                                                                                7s 958ms 
/Users/vanduynslager/.npm-packages/lib
└── [email protected]

$ npm ls -g remark-lint-list-item-indent                                                                                                  8s 118ms 
/Users/vanduynslager/.npm-packages/lib
├─┬ [email protected]
│ └── [email protected] 
└─┬ [email protected]
  └── [email protected] 

[pvdlg]

Does rm -rf node_modules fix anything?

I could try. But I might loose the reproduction env. Plus I'd rather not have to delete my global node_modules...

[wooorm]

Ohhh, I thought stuff was local. I’m guessing global mode interferes with the normal state of things.

Why are all things global?

[pvdlg]

Why are all things global?

Because I'd like to lint any markdown files including:

• ones that in repo that I contribute to that doesn't include remark-lint in their package.json
• ones that that are unrelated to any project / repo. For example when I write some markdown in a file and copy/paste it to a blog post
• ones that are in non npm project (like a README.md in Java project)

[pvdlg]

That said, even in a local install you can't predict that you won't have two version of the same package coexisting. npm or yarn flatten the tree in some circumstances but not necessarly. Plus each new version of npm or yarn can change the flattening algorithm and produce different results.

This is why I proposed to rely on module/rule name rather than on object identity to dedupe.

[wooorm]

There’s no good way of doing this, I think, but you’re the first to raise an issue about it after it existing for a while. And probably because of global installs!

Why not create a .remarkrc in ~, and install everything except remark-cli there? remark-cli will always look for and load ~/.remarkrc if it exists (or a closer one if there is one)!

[pvdlg]

Why not create a .remarkrc in ~, and install everything except remark-cli there? remark-cli will always look for and load ~/.remarkrc if it exists (or a closer one if there is one)!

This is more or less my config. Everything is installed globally and I put .remarkrc in ~.

Are you suggesting to have a package.json in ~ and run npm install from there?

[wooorm]

Yes, or even point from your config file to plugins in some other directory:

// ~/.remarkrc
{
  "plugins": [
    "./some/place/node_modules/remark-preset-lint-recommended",
    ["./some/place/node_modules/remark-lint-list-item-indent", false] // a
    ["./some/place/node_modules/remark-preset-lint-recommended/node_modules/remark-lint-list-item-indent", false] // or b
  ]
}

[pvdlg]

I just tried that:

// ~/.remarkrc
{
  "plugins": [
    "./.npm-packages/lib/node_modules/remark-preset-lint-markdown-style-guide",
    ["./.npm-packages/lib/node_modules/remark-lint-maximum-line-length", false]
  ]
}

~/.npm-packages/lib/node_modules is where my global packages are installed.

It doesn't work, and still report maximum-line-length warning.

[wooorm]

Hmm. 🤔

Does ./.npm-packages/lib/node_modules/remark-lint-maximum-line-length exist?

[pvdlg]

With:

// ~/.remarkrc
{
  "plugins": [
    "~/.npm-packages/lib/node_modules/remark-preset-lint-markdown-style-guide",
    ["~/.npm-packages/lib/node_modules/remark-lint-maximum-line-length", false]
  ]
}

I get the error: Error: Could not find module ~/.npm-packages/lib/node_modules/remark-preset-lint-markdown-style-guide` Even though:

$ ls ~/.npm-packages/lib/node_modules/remark-preset-lint-markdown-style-guide                                                                  400ms → 1 
index.js     node_modules package.json readme.md

[pvdlg]

To make it works I have to configure:

// ~/.remarkrc
{
  "plugins": [
    "./.npm-packages/lib/node_modules/remark-preset-lint-markdown-style-guide",
    ["./.npm-packages/lib/node_modules/remark-preset-lint-markdown-style-guide/node_modules/remark-lint-maximum-line-length", false]
  ]
}

[pvdlg]

Interestingly remark-lint-maximum-line-length doesn't exist anywhere other than in ./.npm-packages/lib/node_modules/remark-preset-lint-markdown-style-guide/node_modules/remark-lint-maximum-line-length

So it doesn't seems to be related to duplicate packages in the tree with different versions.

[wooorm]

No, it’s exactly because global mode doesn’t dedupe!

You can use global mode, but in that case, when reconfiguring plugins, you need to point to the deeper plugin! Global mode doesn’t flatten the tree, which is why it’s not suitable for this.

npm installs stuff differently in ~/.npm-packages than in other places, so you can fix this by creating a normal node_modules somewhere, and use that for your plugins!

[pvdlg]

No, it’s exactly because global mode doesn’t dedupe!

I'm not sure it's a dedupe issue as I have only one time the package remark-lint-maximum-line-length. So when I configure ["remark-lint-maximum-line-length", false] it's necessary loaded from the same place that remark-preset-lint-markdown-style-guide loads it from.

Also does that mean to works locally you need to run npm dedupe? Because neither npm nor yarn systematically dedupe dependencies. As you install new package duplicate will appear (this is why the npm dedupe command exists).

[wooorm]

So here’s what’s going on:

1. remark-lint-maximum-line-length is not remark-preset-lint-markdown-style-guide/node_modules/remark-lint-maximum-line-length
2. The former is turned off, so it isn’t loaded
3. The latter still runs

When I’m talking about deduping I mean the flatting that npm and yarn do automatically. A clean npm install creates that flattened state. Plugins load perfectly if you have that. Global installs don’t do that. You can read more about it in npm’s docs (search for --global-style).

I suggest using remark-cli globally, but don‘t store presets and plugins globally.

[pvdlg]

When I’m talking about deduping I mean the flatting that npm and yarn do automatically. A clean npm install creates that flattened state.

I understand. But as you develop and add new modules this deduped state will not be guaranteed. you would have to delete node_modules and reinstall or run npm dedupe.

I suggest using remark-cli globally, but don‘t store presets and plugins globally.

It's a working workaround indeed.

I'm not sure it's the ideal approach though. It makes it really really hard to lint markdown that is not in a node project. All other linters (eslint, stylelint) do not rely on plugin/rule identity but on names, so this problem doesn't happen. So they can be used globally with globally installed configs and plugin.

[dirkroorda]

I also have a frustratingly hard time to override a rule. I also use remark-cli globally, with a .remarkrc.yaml file in ~. I just wiped all remark related packages from /usr/local/lib/node_modules, and inspected all remaining packages for the presence of remark related packages. No one to be found. I am on the command-line and run remark file.md and my .remarkrc.yaml is

plugins:
  - preset-lint-recommended
  - ['lint-list-item-indent', false]
  - frontmatter
settings:
  verbose: true
  quote: "'"
  quoteSmart: true
  preferUnquoted: true
  setext: true
  closeAtx: true
  bullet: '*'
  emphasis: '*'
  spacedTable: true
  looseTable: true
  paddedTable: false
  listItemIndent: 1

My input file is

http://foo.bar/baz

* item

and this is the output.

<http://foo.bar/baz>

* item
file.md
  1:1-1:19  warning  Don’t use literal URLs without angle brackets  no-literal-urls   remark-lint
       3:3  warning  Incorrect list-item indent: add 2 spaces       list-item-indent  remark-lint

So, the rule list-item-indent is not suppressed. If I say in the plugin section of the config:

plugins:
  - preset-lint-recommended
  - ['lint-list-item-indent', 'tab-size']
  - frontmatter

then I get an error: Error: Could not find module lint-list-item-indent I can npm install it, but it is already inside remark-preset-lint-recommended. If I do install it, and run the command again, lo and behold, I get the warning twice:

<http://foo.bar/baz>

* item
file.md
  1:1-1:19  warning  Don’t use literal URLs without angle brackets  no-literal-urls   remark-lint
       3:3  warning  Incorrect list-item indent: add 2 spaces       list-item-indent  remark-lint
       3:3  warning  Incorrect list-item indent: add 2 spaces       list-item-indent  remark-lint

So there is a serious problem here. I am at a loss how to get a preset to work and then changing one of its rules!

[dirkroorda]

There is a difference between installing rules and activating, using them. The problem now is that a preset installs and activates a number of rules. A later bit of configuration should be able to switch rules on and off without actually installing them. If you enable a rule, but have not installed the rule, you should get a warning.

So my proposal would be: Let the plugin section deal with gathering the installed rules. Let a rule section, outside plugins, declaratively switch rules on and off. A handy option would be

extends: 
  - preset1
  - preset2

which switches on all the rules of mentioned presets, which can be individually overridden by subsequent declarations. I think this is more like how eslint rules work.

An other thing: it would be nice if remark-stringify would look at the same options as the parser.

[OmeGak]

Any update with this issue? I was trying configure stuff with ~/.remarkrc and install plugins globally and got affected by the same issues raised by @pvdlg and @dirkroorda.

I would suggest editing the title of this issue to make it more obvious the problem is related to globally installed plugins.

Also, I understand the proposed workaround of installing plugins locally in ~/node_modules, but is there no plan to fix this properly?

By @pvdlg: This is why I proposed to rely on module/rule name rather than on object identity to dedupe.

@wooorm, would that be too difficult to implement in the current codebase? Would you be against PRs in that direction?

[wooorm]

I would suggest editing the title of this issue to make it more obvious the problem is related to globally installed plugins.

Done, thanks for suggesting!

Also, I understand the proposed workaround of installing plugins locally in ~/node_modules, but is there no plan to fix this properly?

Not really, globally installed plugins is an anti-pattern, as global stuff is meant to be a top-level thing, not to be resolved and used.

This is why I proposed to rely on module/rule name rather than on object identity to dedupe. would that be too difficult to implement in the current codebase? Would you be against PRs in that direction?

This would mean that about ±200 plugins, often from the community, would need to be updated, that we can’t rely on node’s module caching (because we load different plugins, but pick one of them because they share an identifier), and that those identifiers also need to change when a new version is published.

[dietergeerts]

This is why I proposed to rely on module/rule name rather than on object identity to dedupe.

There’s no good way of doing this, I think, but you’re the first to raise an issue about it after it existing for a while. And probably because of global installs!

Relying on module/rule name is the better solution, to not have conflicts with versions of the rule. Eslint does this too, and rules are named according to ${package}/${rule}, and it works very well.

I'm currently trying to use some presets together, and the one overwrites some rule of the other, and that's fine. It's only very strange how remark-lint is setup, as each preset needs to include remark-lint, making it too tightly coupled. A preset should define the rules, and have a peer-dependency on remark-lint, no? It will make so much more sense. Presets should also have a peer-dependency on the plugins they use. Only in this way will versions be installed once with the one defined by the consumer. It is a bit more management for the consumer, but less fragile and then you have a good overview of the things you are using.