berry icon indicating copy to clipboard operation
berry copied to clipboard

[Feature] Lock binaries generated in build scripts.

Open seivan opened this issue 2 years ago • 3 comments

  • [x] I'd be willing to implement this feature (contributing guide)
  • [x] This feature is important to have in this repository; a contrib plugin wouldn't do

Edit: Running canary at RC11

cacheFolder: ./dependencies
cloneConcurrency: 18
compressionLevel: 9
enableGlobalCache: false
enableHyperlinks: true
enableInlineBuilds: false
enableTelemetry: false
installStatePath: ./dependencies/install-state.gz
networkConcurrency: 50
nodeLinker: pnp
plugins:
  - path: .yarn/plugins/@yarnpkg/plugin-outdated.cjs
    spec: "https://mskelton.dev/yarn-outdated/v3"
  - path: .yarn/plugins/plugin-envs.cjs
    spec: "https://raw.githubusercontent.com/Ayc0/yarn-plugin-envs/v0.0.3/src/index.js"
pnpEnableInlining: true
pnpMode: strict
pnpUnpluggedFolder: ./dependencies/unplugged
supportedArchitectures:
  cpu:
    - arm64
    - x64
  os:
    - darwin
    - linux
yarnPath: .yarn/releases/yarn-4.0.0-rc.11.cjs
.yarn/*
!.yarn/cache
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions

Describe the user story

Is there a way to "lock" down binaries and executables generated/download from dependencies during their post-install build scripts?

Gotten into the rhythm of using the arch matrix defined above to zero install things like esbuild for everyone, including CI. However, I've noticed that running yarn install --immutable --inline-builds seems to re-generate those, which is a bit unnecessary given that the version in the manifest (package.json) hasn't changed.

Describe the solution you'd like Was hoping there was a way in package.json to tell it, not to run build scripts for a specific dependency until its version has actually changed or arch/cpu matrix was modified.

I got

  "dependenciesMeta": {
    "core-js": {
      "built": false
    }
  }

But that's for every version and because it's spamming, not an actual build. Would be neat if we could add something like

  "dependenciesMeta": {
    "esbuild": {
      "built": "version" 
    }
  }

Where built: false as long as the version in the lockfile hasn't changed.

Describe the drawbacks of your solution I am not 100% sure it's an actual solution or the cause of the problem. Another drawback is that some this duplicates "versioning" in two places. Once for the package itself, and once for "Locking" its build script down, might be a bit complicated implementation wise.

Describe alternatives you've considered

Manually toggling it off like described above, but this will be error prone.

Related: https://github.com/yarnpkg/berry/issues/4366

seivan avatar Jul 05 '22 07:07 seivan

Actually... I think the problem stems from these:

cacheFolder: ./dependencies
pnpUnpluggedFolder: ./dependencies/unplugged

being changed.

I would always get a ➤ YN0019: │ unplugged appears to be unused - removing, even though it didn't remove) There might be a bug related to non-default location settings at first glance.

seivan avatar Jul 05 '22 16:07 seivan

Make sure the pnpUnpluggedFolder isn't in the same folder as cacheFolder. You can try the following:

cacheFolder: ./dependencies/cache
pnpUnpluggedFolder: ./dependencies/unplugged

merceyz avatar Jul 05 '22 18:07 merceyz

Make sure the pnpUnpluggedFolder isn't in the same folder as cacheFolder. You can try the following:

Thank you, I could have sworn the documentation never mentioned this as an issue.

cacheFolder: ./dependencies/cache
pnpUnpluggedFolder: ./dependencies/unplugged

Again, thanks. You seem to understand what I was trying to go for.

Just out curiosity, do you know if there are actual differences between keeping things inside .yarn and doing what you suggested?

My take is, if the config supports moving them, then it should really support it, but if it comes with compromises, then it might not be worth it.

seivan avatar Jul 06 '22 02:07 seivan