Remove "engines" fields from all package.json files for improved backwards compatibility with unmaintained package managers

[jacobp100]

Is it possible to better support Yarn v1?

I speak from having the exact same experience at my last 6 or 7 companies: they don't want to use NPM because it has a lot of bugs throughout the versions. They all use Yarn 1, and almost all of them also faced this Yarn bug stopping them running yarn add. Everyone who did face it just ignores it by manually adding packages to the package.json, then running yarn. There is no appetite for upgrading to yarn 3

The docs suggest yarn add sharp --ignore-engines. As mentioned earlier, yarn add doesn't work for a lot of people

Happy to help out if some context is provided why yarn 3 was put as the minimum version

(Node v20.11.0)

[lovell]

Please ensure you've read and understood https://github.com/lovell/sharp/issues/3750 for context.

I continue to recommend that those who wish to use yarn as their package manager should upgrade to its latest version. https://mastodon.social/@lovell/111726943907221407

("They all use Yarn 1, and almost all of them also faced this Yarn bug" and "they don't want to use NPM because it has a lot of bugs" is quite the juxtaposition :sweat_smile: .)

[jacobp100]

Thanks for the fast reply 😄 I certainly remember NPM issues like packages magically updating themselves when you install new packages, and it was never really possible to stop it happening. I will say I never questioned why people use yarn since I prefer it anyway. To emphasise, I didn't set up Yarn 1 at these companies, that's what they were using before I joined. I was contracting for quite a few years, I've only seen one company using something other than Yarn 1 since 2015. It's still really popular in industry. I think the reality is Yarn is facing it's Python 3 moment, and I'd be surprised if it wasn't the same situation in a decade 😅

I did find and read that PR via git blame etc. - but I can't really work out why the version is set to 3. It seems like it works with yarn 1 - as long as that flag is provided - and the only issue is that the minimum engine is set higher than it needs to be. At least locally, I only have the binaries that are strictly necessary. What are the actual issues stopping the engine requirement being lower?

[lovell]

I did a quick search for the sort of error message that yarn v1 users might see to assess the impact:

https://github.com/search?q=%22The+engine+glibc+appears+to+be+invalid%22&type=issues

Clearly this is causing some pain for other open source maintainers, so for that reason let's see what we can do.

I think the easiest thing to do is remove the engines field entirely (we'll need to move the glibc and macos keys somewhere else) and update the docs. Leave this with me.

I'm still of the opinion that the sooner someone presses the official end-of-life button on yarn v1, the better. The maintainers of the official Node Docker images are considering removing it - see https://github.com/nodejs/docker-node/issues/1979

[lovell]

Work-in-progress for this is at https://github.com/lovell/sharp/commit/7146a9cc5f5761ca9446f4aa3f93675d0bf2981f but it will depend on https://github.com/lovell/sharp-libvips/commit/459b35f0b8b67dd3ae0bec9dc27ec6914f92981f and newer, not-yet-published versions of the @img/sharp-libvips-* packages.

[lovell]

v0.33.5 now available using only values for the engines property that yarn v1 is known to handle.

[jacobp100]

Thank you! 🤩

[jacobp100]

@lovell is the expectation for people on yarn v1 that they add the pre-built packages to their package.json? I'm firstly getting warnings like

warning @img/[email protected]: The engine "pnpm" appears to be invalid.

Followed by

Could not load the "sharp" module using the linux-x64 runtime ERR_DLOPEN_FAILED: libvips-cpp.so.42: cannot open shared object file: No such file or directory Possible solutions:

• Ensure optional dependencies can be installed: npm install --include=optional sharp
• Ensure your package manager supports multi-platform installation: See https://sharp.pixelplumbing.com/install#cross-platform
• Add platform-specific dependencies: npm install --os=linux --cpu=x64 sharp
• Consult the installation documentation: See https://sharp.pixelplumbing.com/install

It's not problem for us to add these packages - but I was wondering if that's the expected way to do things now

[lovell]

warning @img/[email protected]: The engine "pnpm" appears to be invalid.

Please upgrade to v0.33.5

[jacobp100]

That was the fix! Some other package was pulling in an older version, managed to resolve it

[karlhorky]

Interesting issue 👀

For those projects who still have not yet managed to migrate away from the legacy Yarn v1 version, does https://github.com/lovell/sharp-libvips/commit/459b35f0b8b67dd3ae0bec9dc27ec6914f92981f and the [email protected] release also represent a reversal of the need for --ignore-engines when running yarn install / yarn?

Right now, in such projects, we're always running yarn --ignore-engines, to avoid a broken sharp package.

We plan to migrate from Yarn v1 to pnpm, but haven't gotten around to it in all repos yet.

[lovell]

sharp v0.33.5 removes the need for --ignore-engines but please don't tell your friends, please continue to migrate away from yarn v1 if only to make the lives of open source maintainers easier.

[karlhorky]

Definitely agreed that Yarn v1 needs to be migrated away from - companies and projects still stuck on it need to invest.

Probably would even be good to see more hard incompatibilities (no workarounds) to motivate companies to switch faster.