Can't install scoped packages from an upstream registry

[jameslnewell]

I can install and publish scoped packages to a Sinopia registry but can't install scoped packages from an upstream repository (e.g. http://registry.npmjs.com/).

Sinopia doesn't seem to handle URLs with @ or %2f correctly but the global registry does:

http://localhost:4873/@digitaledgeit%2fpaypal => 404
http://registry.npmjs.com/@digitaledgeit%2fpaypal => 200

---

Platform:

OS: Windows 7
Node: v0.12.6
Sinopia: 1.4.0

Server:

c:\tmp\node_modules\sinopia>node bin\sinopia
 warn  --- config file  - C:\Users\jnewell\AppData\Roaming\sinopia\config.yaml
 warn  --- http address - http://localhost:4873/
 http  <-- 404, user: undefined, req: 'GET /@digitaledgeit%2fpaypal', error: no such package available

Client:

PS C:\tmp> npm i @digitaledgeit/paypal --registry=http://localhost:4873
npm ERR! Windows_NT 6.1.7601
npm ERR! argv "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program     Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "i" "@digitaledgeit/paypal" "--registry=http://localhost:4873"
npm ERR! node v0.12.6
npm ERR! npm  v2.11.2
npm ERR! code E404

npm ERR! 404 no such package available : @digitaledgeit/paypal
npm ERR! 404
npm ERR! 404 '@digitaledgeit/paypal' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! Please include the following file with any support request:
npm ERR!     C:\tmp\npm-debug.log

Apologies, I'm not sure if this is a duplicate of #104.

[jameslnewell]

OK. I figured out the default config doesn't proxy scoped packages to the npmjs registry and have updated my config to do so:

packages:
  '@*/*':
    # scoped packages
    access: $all
    publish: $authenticated
    proxy: npmjs # <==========

Now I'm getting:

http  --> 404, req: 'GET https://registry.npmjs.org/%40digitaledgeit%2Fpaypal', bytes: 0/81
http  <-- 404, user: undefined, req: 'GET /@digitaledgeit%2fpaypal', error: no such package availab

The problem seems to be the encode method (encodeURIComponent) in up-storage.js which encodes the @ symbol to %40 but npmjs prefers the non-encoded version.

[david-gang]

Hi @jameslnewell I try to fix the issue with rxjs and as #280

var encode         = function(thing) {
  return encodeURIComponent(thing).replace(/^%40/, '@');
};

does not work.

Do we need also

var encode         = function(thing) {
  return encodeURIComponent(thing).replace(/^%40/, '@').replace('/', '%2f');
};

?

[jameslnewell]

Nah, the / is part of the name and needs to be encoded otherwise the registry thinks you've hit the wrong route and you get a 404.

http://registry.npmjs.com/@digitaledgeit/paypal
http://registry.npmjs.com/@digitaledgeit%2fpaypal

I'm not sure why the ^ is in the regular expression? You could try removing that.

[scic]

I can confirm that pulling scoped packages through sinopia currently does not work with these fixes. It would be great if support for that would be added.

[rollercodester]

Ran into same issue. Was able to workaround by specifically setting my npm private repository scope via npm config.

E.g. npm config set @myscope:registry https://registry.npmjs.org/

Works like a charm. All @myscope libs are pulled straight from npm without going thru sinopia and all other libs are pulled from sinopia as desired.

[fabiosantoscode]

@rollercodester I've also been using that workaround for great effect :)

+1 on support from me too. Having problems with the @semantic-release scope.

[scottrippey]

@rollercodester Thanks for that workaround! Worked for me.

For anyone also having trouble with @semantic-release, getting an error message like XXX is not allowed to access package @semantic-release/commit-analyzer, here is the workaround:

Either run npm config set @semantic-release:registry https://registry.npmjs.org/

Or add this to your .npmrc file:

@semantic-release:registry=https://registry.npmjs.org/

Hope that helps!

[vukasin-nikodijevic]

any progress on this ?

@ionic @angular @types all of them are scoped and they are very important to be loaded from cache as they are big portion of packages to fetch for any angular based project.

[ghost]

Hi guys,

I'm not sure if this issue has been resolved yet, but after struggling for a few hours to get this working on v1.4.0 I managed to get a fix based upon the comment from @david-gang. I just simplified it a bit and somehow it seems to be working without the need to make changes to npm's config...

var encode = function(thing) { return encodeURIComponent(thing).replace('%40', '@'); };

This was simply added to the top of 'up-storage.js'. Not the prettiest solution, but it works...

I unfortunately do not have much time to test and/or submit a PR, but if someone is stuck on the same problem I believe they may simply make the changes to their installed version... This solves the issue of installing packages from an upstream registry, but I do not have any local scoped packages to test with, so I am not sure how they will be effected...

[jameslnewell]

@detachedJD that's what the merged PR does https://github.com/rlidwka/sinopia/pull/280/files

The problem is likely that sinopia hasn't been released since Jun 2015 and the PR was merged in Sep 2015.

[romandecker]

Ok, is sinopia still under active maintenance (i.e. being published to npm on a regular basis)? Because we just ran into this issue today and we had to workaround it by adding the scope to all our .npmrc-files.

[petershaw]

If sinopia is no longer under active development, what are you guys using next?

[juanpicado]

@petershaw use https://github.com/verdaccio/verdaccio . A sinopia fork

[vukasin-nikodijevic]

@petershaw - you should move to yarn

[tsanhan]

can someone please explain how to install any packages from an upstream registry to verdaccio/sinopia? please!!

[juanpicado]

Do you mean cache packages? Your question is a bit generic.

[tsanhan]

yes. is there other form of understanding the question. I'm sorry but i'm new to all this. i'm trying to setup a local registry with packages from npmjs.org and i'm not sure how to do it.

thanks.

[juanpicado]

@tsanhan with verdaccio comes by default, I guess is the same with sinopia. You do not have nothing to change unless you want add more registries. You can read the verdaccio doc which is quite the identical as sinopia at this point of time. https://github.com/verdaccio/verdaccio/tree/master/wiki

[tsanhan]

@juanpicado, i need no more registries. but I cant find a way to install npmjs.org packages on it!, that's it! can you please direct me on how to accomplish this? te end result need to be a listing if my node_moduls packages in verdaccio server (listed in localhost:4873)

[Seungwoo321]

I fixed config.yaml and solved. And have to restart sinopia.

Add "proxy: npmjs" in packages: '@*/*':

packages:
  '@*/*':
    # scoped packages  
    access: $all
    publish: $authenticated
    proxy: npmjs <--- add 

[GuiToniello]

@Seungwoo321 For me, it did the trick ! Thank you :)