patch-package
patch-package copied to clipboard
ds300
make work with pnpm and git, use pnpm-lock.yaml
part of issue https://github.com/ds300/patch-package/issues/35#issuecomment-691512252
error was
**ERROR** No package-lock.json, npm-shrinkwrap.json, or yarn.lock file.
install
to install my patched version of patch-package
pnpm install @milahu/patch-package-with-pnpm-support
npx patch-package
why is this PR still open?
the author of patch-package has abandoned the project,
so nobody has write access to the repo. feel free to fork my fork
changes
- use
pnpm-lock.yamlto get the resolvedVersion - use pnpm to install the origin package (if
pnpm-lock.yamlwas found) - workaround for pnpm symlinks
- add some debug output, also show the output of
npm installetc (todo: add a--verboseflag to CLI) - exclude lockfiles from diff. maybe exclude more files with
patch-package --exclude '^dist/.*' - fix some lint errors via
tslint --fixandprettier --write(sorry for the diff noise) - allow comment header in patch files, see my comment
- support pnpm's
link:../../../some/relative/path- detect local git repos
- patch is based on the
origin/HEADcommit - use full commit hash as version for npm
- as fallback, use version from local
package.json
- patch is based on the
- dont copy
node_packagesfolders from local package
- detect local git repos
todo
are git patches applied correctly? we must install the exact commit
fixme
when we install packages from git (pinned to commit), patch-package will say
Warning: patch-package detected a patch file version mismatch
cos the patch version is the "declared" version from /package.json = commit hash
= for example 8a5162785c58864ef65308c0f0d890cade0b407f
and the "applied to" version ("resolved" version) is from node_modules/pkg/package.json
= for example 1.0.0-canary.10
both versions (declared and resolved) could be stored in the comment header of the patch file
oof. tests pass locally but fail on ci cos (snapshots && snapshots.length) == null
maybe the *.sh scripts run in posix shell?
edit: yepp, removed shopt -s expand_aliases
@milahu Awesome work! I was just about to look into this myself when I found your PR.
thx for the keepalive ; )
thing is, the main patch-package repo is slightly undermaintained ...
i mean, my last patch (#252) was 3 lines, and that took 190 days to merge.
looks like ds300 is the only maintainer and shifts the backlog only like 2 or 3 times per year : /
i never really came to use my patch, let me catch up on that:
install patch-package branch pnpm-support-2
cd /my/fancy/node/project
mkdir patched_packages
cd patched_packages
git clone --depth 1 https://github.com/milahu/patch-package.git --branch pnpm-support-2
cd patch-package
pnpm install
pnpm run build
cd ../../
node patched_modules/patch-package/dist/index.js some-node-package
tests
edit: github CI says tests passed 0__o let me run tests again ... edit 2: nope, still the same error:
# if you have some minutes to waste ...
pnpm run test
# jest --runInBand
# ts-jest[config] (WARN) Unable to find the root of the project where ts-jest has been installed.
# ts-jest[versions] (WARN) Version 4.2.4 of typescript installed has not been tested with ts-jest. If you're experiencing issues, consider using a supported version (>=2.7.0 <4.0.0). Please do not report issues in ts-jest if you are using unsupported versions.
# FAIL src/packageIsDevDependency.test.ts
# ● packageIsDevDependency › returns false if package is a transitive dependency of a dev dependency
# [ may be related to my patch ]
# Test Suites: 1 failed, 35 passed, 36 total
# Tests: 1 failed, 5 skipped, 804 passed, 810 total
# Snapshots: 65 passed, 65 total
# Time: 438.918s
one problem i found:
when /package.json says .dependencies.some-package = link:../../some/where
then /pnpm-lock.yaml says .dependencies.some-package = 1.2.3
so the "resolved" version (from /pnpm-lock.yaml) is wrong
and my "detect git version" logic in patch-package/getPackageResolution.js is not used
-> added a workaround
another problem, also
when
/package.jsonsays.dependencies.some-package=link:../../some/where
... then that /package.json will obviously not work on other machines (not portable)
and we would have to replace link:../../some/where
with the git-version (or npm-version), that the patch is based on
or straight-up throw a fatal error in patch-package when /package.json says link:../../some/where?
opinions on this? what is a "reasonable default" workflow to combine git and pnpm?
when i do pnpm install git+https://github.com/ds300/patch-package.git
then node_modules/patch-package is not a git repo
when i do
(
cd /tmp; git clone --depth 1 https://github.com/ds300/patch-package.git
cd patch-package; pnpm install; pnpm run build
)
pnpm install /tmp/patch-package
then the resolvedVersion in /pnpm-lock.yaml is wrong (not the git version)
and /package.json is not portable
so ... imho the only solution is to make pnpm clone the git repo
by running something like pnpm install --git-clone git+https://github.com/ds300/patch-package.git
this would clone with a default depth of 1, allowing to set --git-depth <depth>
ideally, following runs of pnpm install or pnpm remove should not change the git repo
when
/package.jsonsays.dependencies.some-package=link:../../some/where... then that
/package.jsonwill obviously not work on other machines (not portable) and we would have to replacelink:../../some/wherewith the git-version (or npm-version), that the patch is based on or straight-up throw a fatal error in patch-package when/package.jsonsayslink:../../some/where?opinions on this? what is a "reasonable default" workflow to combine git and pnpm?
I feel like using patch-package on a locally linked package is way too niche, and if it takes way too much work or maintenance in the future just to keep it working, it should just throw, I'm pretty sure the 99% of the usage of patch-package is just for npm packages.
it should just throw
agree, linked packages are not portable
I'm pretty sure the 99% of the usage of patch-package is just for npm packages.
... but some changes are non-trivial for example when i want to cherry-pick some unmerged PRs (i usually create PRs for my patches) but i want to publish my "private merge" branch via patch-package, not via git
my workflow can be solved with
1. in the npm project, install some-package from git. pin the version with the full commit hash
we can install the latest version from npm, but then the diff is larger
2. in a separate folder, clone the git repo of some-package, make changes, branches, commits, ...
3. copy the patched files to the npm project's node_packages/some-package/
sample: rsync --archive /tmp/repo/some-package/ /tmp/project/node_packages/some-package/
4. run patch-package some-package
step 3 could be automated with a file watcher, e.g. chokidar.watch
another detail (see next comment)
the version of the patched package should be pinned in /package.json (single source of truth)
so it works with all package managers at all times
done: restore the old getPackageVersion for file protocol (to fix tests),
and for other protocols (http://some/file.tgz#sha1sumoftgzfile, etc.)
when the package is installed from git, patch-package will require to pin the version in package.json:
{
"dependencies": {
"some-package": "git://some/where#fullcommithash"
}
}
todo: automate the version-pinning with a --pin-version flag for patch-package
-> will edit package.json to pin the latest version
discuss: is this "version-pinning in package.json" too pedantic?
should we "just create a patch" and complain later on version mismatch?
currently applyPatches.ts will say
Warning: patch-package detected a patch file version mismatch
todo: fix installedPackageVersion in applyPatches.ts -> should use the commit hash for git repos
either from declaredVersion, or from git rev-parse HEAD
fixed: the --debug flag consumed the next argv
replaced the unmaintained minimist argv parser with dashdash
tests: to test npm packages, we could run a local npm registry
done: add header comments to the generated patch files. sample:
# generated by patch-package 6.4.7 on 2021-05-12 20:07:53
#
# command:
# npx patch-package --exclude '^test/stubs.*' @11ty/eleventy
#
# declared package:
# @11ty/eleventy: github:11ty/eleventy#8a5162785c58864ef65308c0f0d890cade0b407f
#
# header comments also make room for custom "patch messages", like ...
#
# this patch will
#
# * add feature xyz
# * fix bug 123
#
diff --git a/node_modules/@11ty/eleventy/src/Engines/Txt.js b/node_modules/@11ty/eleventy/src/Engines/Txt.js
new file mode 100644
index 0000000..aef38ca
--- /dev/null
+++ b/node_modules/@11ty/eleventy/src/Engines/Txt.js
@@ -0,0 +1,34 @@
+// based on Html.js
+
+const TemplateEngine = require("./TemplateEngine");
...
todo: when package is a git repo, add commit messages as diff header comments
todo: when an old patch exists, prepend the old header comments to the new patch
done: add prepare script to auto-build on install from source
pnpm i github:milahu/patch-package#pnpm-support-2 # requires pnpm 6.2.2
npx patch-package
todo: emulate git status to preview what files will be changed or added -> exclude unwanted files before makePatch
so ... maintainer is gone, nobody wants to maintain a fork, so we must install patch-package from some hidden fork branch, cos we cant use patch-package to patch patch-package : D
pnpm i github:milahu/patch-package#pnpm-support-2 # requires pnpm 6.2.2 npx patch-package
its not perfect, but it works : )
Does anybody know if the pnpm folks would be willing to incorporate this directly into pnpm (similar to Yarn 2)?
Thanks for keeping this branch alive. It's a huge help. I did have to take away all of the wildcard selectors in my package.json and pin the exact versions, which isn't great, but at least it worked!
🤞 hopefully the maintainer can give this a quick look. Pretty please @ds300
This works well in our project though I hope it can also support workspace so I don't have to install the package at the top level. Hope this can be merged soon
I did have to take away all of the wildcard selectors in my
package.jsonand pin the exact versions
why? whats your package manager?
I hope it can also support workspace
PRs welcome
why? whats your package manager?
pnpm 6.13.0, this is w/ an extremely basic project. I think your PR should be merged without changes because right now nothing works. Was just writing it up in case someone else came across the workaround.
right now nothing works
cannot reproduce
cd $(mktemp -d)
npm init -y
pnpm i github:milahu/patch-package#pnpm-support-2 # requires pnpm 6.2.2
npx patch-package
ouput
patch-package 6.4.7
Applying patches...
No patch files found
env
$ npm --version
8.1.0
$ pnpm --version
6.22.2
Well, uhm, you gotta install a dependency and try to patch package it.
Steps to reproduce
- Download patch-package from your nice PR branch
- Install any dependency (by default uses wildcard notation)
- Run patch-package
cd $(mktemp -d)
npm init -y
pnpm i github:milahu/patch-package#pnpm-support-2 # requires pnpm 6.2.2
npx patch-package
pnpm install lodash # Writes ^14.XX.XX to package.json
pnpm patch-package lodash
And you'll get...
➜ tmp.SLmTyTJs pnpm patch-package lodash patch-package 6.4.7
• Creating temporary folder
Error: error: found wildcard version. package.json must pin the exact version of lodash in the format <package>@<major>.<minor>.<patch>
And the full stacktrace....
Like my comment says, once you remove the wildcard notation by swapping the lodash version from ^14.XX.XX => 14.XX.XX (like the error message says) the error disappears.
Versions:
pnpm 6.13.0
npm 6.14.13 # Also fails on 8.1.0
node v14.17.0 # Also fails on v16.14.0
OSX Big Sur, 11.3
Ohhh. I understand why you were annoyed.
right now nothing works.
with the existing repo/project on main. Your PR works fine, minus the need to pin all of the dependencies. Regardless of if you fix the bug I posted above, your PR at least gives some sort of workaround.
thanks for reporting, fixed
# binary release
pnpm install @milahu/[email protected]
# source release
pnpm i github:milahu/patch-package-with-pnpm-support
pnpm i github:milahu/patch-package#pnpm-support-2
note: some tests are failing
patch-package != pnpm
what's the hold up here ?
patch-package is unmaintained.
feel free to use my pnpm install @milahu/patch-package-with-pnpm-support
and/or maintain your own fork of patch-package
if you allow upstream packages to block your workflow, youre doing something wrong
patch-package is unmaintained. feel free to use my
pnpm install @milahu/patch-package-with-pnpm-supportand/or maintain your own fork of patch-packageif you allow upstream packages to block your workflow, youre doing something wrong
Thank you! Any reason why you didn't merge it into main of your fork of the patch-package? :)
this PR is merged in https://github.com/milahu/patch-package-with-pnpm-support
Hey @milahu
thanks for your work
this PR is merged in https://github.com/milahu/patch-package-with-pnpm-support
I tried to use it in mono-repo setup but this does not work
if I install at root and execute pnpm patch-package <package-to-patch> it cannot find the package
If I instead install it on the sub folder, it cannot find pnpm-lock.yaml
thanks for reporting. pnpm workspaces (monorepos) are not implemented should be easy to do ... please dont wait for me, i have no time for this see also #277 and yarn workspaces and npmv7 workspaces
@milahu I found a fork of your repo that adds the pnpm lock file detection for workspaces... in case that is something you could easily roll back into your repo.
https://github.com/Tellus/patch-package-with-pnpm-support
thanks @Tellus thanks @caschbre
published as @milahu/patch-package-with-pnpm-support version 6.4.9
thanks @Tellus
You're perfectly welcome. I mulled issuing a pull request back then but wasn't sure if it was up to code standards.
Glad to have been of help :)
Thanks for the great work on the package in general!
renamed to @milahu/patch-package
@milahu/patch-package-with-pnpm-support is deprecated
How do people use patch-package with pnpm for a dependency of a workspace package?
Good news!! PNPM now has official support for yarn style patch!
https://github.com/pnpm/pnpm/issues/2675
We just need to wait for its next release. The benefit of the official support in PNPM is that you won't corrupt the global store (they're shared!)
Also thanks for @milahu, your PR helped us go through the hard time before the official support!
