Does not work with Yarn v2 (berry)

[EliteMasterEric]

While using Yarn version 2, an exception occurs with the example command (or any other repo):

> yarn add https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello

Internal Error: Invalid descriptor (https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello)
    at Module.x (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:434115)
    at D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:29878
    at Array.map (<anonymous>)
    at M.execute (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:29786)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async M.validateAndExecute (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:659490)
    at async Y.run (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:17:3854)
    at async Y.runExit (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:17:4021)
    at async h (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:284223)
    at async r (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:282834)

If the command required for gitpkg must be modified, the website should be updated to include this command alongside yarn v1 and npm.

[EqualMa]

I am sorry that I haven't tried out Yarn 2 yet, but have you tried

yarn add https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello.tgz

, with .tgz appending to the url?

[bryanvaz]

I am sorry that I haven't tried out Yarn 2 yet, but have you tried

yarn add https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello.tgz

, with .tgz appending to the url?

Hi @EqualMa ,

I have been trying to get the app to work on my own vercel deployment so I can submit a PR to fix this for Yarn 2 but it just refuses to work. Your example above works, however it does not work for a commit referenced subfolder. Specifically I am testing with the following url

• https://gitpkg.now.sh/aws/aws-xray-sdk-node/packages/mysql?811d9582103cafcf948dade3efaef1b92aa211bd - which does work but is incompatible with Yarn 2
• https://gitpkg.now.sh/aws/aws-xray-sdk-node/packages/mysql.tgz?811d9582103cafcf948dade3efaef1b92aa211bd - returns an empty file, but is Yarn 2 compatible.

Are you able to see the logs on your side and figure out why the function is returning an empty file?

Attached to yarnpkg/berry#2729

Thanks, Bryan

[EliteMasterEric]

Hey @bryanvaz,

I wanted to let you know that in my project, I have found a workaround for this issue.

I have created a packages folder, and for each package I was including from a Git repository, I added it as a submodule. I then configured yarn to install with the URL file:./packages/<NAME>, and also added it to the resolutions section so other modules would use my version as a sub-dependency.

Hope this helps you.

[bryanvaz]

Hi @MasterEric, I ended up doing something similar. I just create a separate Github for the package, then push only the package I want to Github.

Since Yarn 2 requires .tgz or .tar.gz in the path (but it can be anywhere), I would like to try to update your function so it can work with this stupid requirement. (Your function is super useful because everyone from babel to even yarn are using monorepos now.) It should just be as simple as updating your regex to detect and filter out a .tgz and .tar.gz extension from the url.

The basic fix I came up with (because I cannot test changes to the regex) is: https://github.com/EqualMa/gitpkg/blob/3329405dd0b6a03814fce2ba78924bccdd75b9f5/packages/core/src/parse-url-query/plugins/url-and-commit/from-url.ts#L24-L26 And change it to:

const { url: u, commit: c } = res.params; 
const url = decodeURIComponent(u.replace(/((\.tgz)|(\.tar.gz))$/,'')); 
const commit = c && decodeURIComponent(c); 

If you have the time, can you apply it to your function and test to see if it works?

If so, it should make GitPkg compatible with Yarn 2 and Yarn 3.

Thanks, Bryan

[EqualMa]

Thanks a lot for your tests and contributions! I just found the formal api should work for yarn2 but it had been wrong documented, so I just made a commit to fix the docs. Now you can refer to docs for formal api.

For the package mentioned above, the url will be:

https://gitpkg.vercel.app/api/pkg.tgz?url=aws/aws-xray-sdk-node/packages/mysql&commit=811d9582103cafcf948dade3efaef1b92aa211bd

Please help to test this.

As for the simplest api, I think your changes should work but I don't have time to test it. So I will leave this issue open for now.

[rafifos]

Hi! Just found out gitpkg and the idea seems awesome.

Even after updating the documentation, I can't get it to work in the Next.js monorepo.

yarn add --dev "https://gitpkg.now.sh/api/pkg.tgz?url=housseindjirdeh/next.js/packages/e
slint-config-next&commit=eslint"

Internal Error: Invalid descriptor (https://gitpkg.now.sh/api/pkg.tgz?url=housseindjirdeh/next.js/packages/eslint-config-next&commit=eslint)

[mamiller93]

@rafifos did you ever get this working? I'm running into a similar issue with yarn 2.4.1

[EqualMa]

@rafifos It seems that yarn requires the uri component to be encoded strictly, you can try https://gitpkg.now.sh/api/pkg.tgz?url=housseindjirdeh%2Fnext.js%2Fpackages%2Feslint-config-next&commit=eslint

[evelant]

I can confirm that the way to work with newer yarn versions is to follow this format for the URL:

https://gitpkg.now.sh/api/pkg.tgz?url= + URI encoded path + &commit= + commit specifier