deno icon indicating copy to clipboard operation
deno copied to clipboard

Unable to install npm package from private registry

Open bavoco opened this issue 1 year ago • 8 comments

Version: Deno 2.0.0-rc.7 OS: Windows 11

package.json

{
  "name": "repro",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {},
  "type": "module",
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@myscope/packagename": "^14.6.2"
  }
}

deno.json

{}

.npmrc

@myscope:registry=https://some.gitlab.server.com/api/v4/packages/npm/
//some.gitlab.server.com/api/v4/packages/npm/:_authToken=supersecret

When I run deno install, the output is:

error: Failed caching npm package '@myscope/[email protected]'.

Caused by:
    Could not find npm package tarball at: https://some.gitlab.server.com/api/v4/projects/12345/packages/npm/@myscope/packagename/-/@myscope/packagename-14.6.2.tgz

The resulting node_modules folder looks like: image

Notice that it did download the dependencies of the package I was trying to install. So although it says it could not find a package tarball, it definitely did find something at the url and it is using the authtoken from the npmrc correcly otherwise it would have had a 404.

The output of `DENO_DIR=C:/TEMP/deno DENO_LOG=debug deno install` is:
DEBUG RS - deno_config::deno_json:780 - Config file found at 'C:\Users\me.me\Downloads\repro\deno.json'
DEBUG RS - deno_config::workspace::discovery:263 - package.json file found at 'C:\Users\me.me\Downloads\repro\package.json'
DEBUG RS - deno::args:931 - Finished config loading.
DEBUG RS - deno::npm::managed::resolution:282 - Running npm resolution.
DEBUG RS - hyper_util::client::legacy::connect::dns:122 - resolving host="some.gitlab.server.com"
DEBUG RS - hyper_util::client::legacy::connect::http:643 - connecting to 10.170.97.45:443
DEBUG RS - hyper_util::client::legacy::connect::http:646 - connected to 10.170.97.45:443
DEBUG RS - h2::client:1281 - binding client connection
DEBUG RS - h2::client:1286 - client connection bound
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
DEBUG RS - h2::proto::connection:138 - Connection; peer=Client
DEBUG RS - hyper_util::client::legacy::pool:396 - pooling idle connection for ("https", some.gitlab.server.com)
DEBUG RS - h2::codec::framed_write:213 - send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 }
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) }                           
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x0), max_concurrent_streams: 128, initial_window_size: 65536, max_frame_size: 16777215 }
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::codec::framed_read:405 - received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 2147418112 }
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::proto::settings:56 - received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) }
DEBUG RS - deno_npm::resolution::graph:927 - <package-req> - Resolved @myscope/packagename@^14.6.2 to @myscope/[email protected]
DEBUG RS - hyper_util::client::legacy::connect::dns:122 - resolving host="registry.npmjs.org"
DEBUG RS - hyper_util::client::legacy::connect::dns:122 - resolving host="registry.npmjs.org"
DEBUG RS - hyper_util::client::legacy::connect::http:643 - connecting to 104.16.31.34:443
DEBUG RS - hyper_util::client::legacy::connect::http:643 - connecting to 104.16.31.34:443                                                                
DEBUG RS - hyper_util::client::legacy::connect::http:646 - connected to 104.16.31.34:443                                                                 
DEBUG RS - hyper_util::client::legacy::connect::http:646 - connected to 104.16.31.34:443                                                                 
DEBUG RS - h2::client:1281 - binding client connection
DEBUG RS - h2::client:1286 - client connection bound
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
DEBUG RS - h2::proto::connection:138 - Connection; peer=Client
DEBUG RS - hyper_util::client::legacy::pool:396 - pooling idle connection for ("https", registry.npmjs.org)
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x0), max_concurrent_streams: 100, initial_window_size: 65536, max_frame_size: 16777215 }
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::codec::framed_read:405 - received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 2147418112 }
DEBUG RS - h2::codec::framed_write:213 - send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 }
DEBUG RS - hyper_util::client::legacy::pool:270 - reuse idle connection for ("https", registry.npmjs.org)
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) }
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(3), flags: (0x5: END_HEADERS | END_STREAM) }
DEBUG RS - h2::client:1281 - binding client connection                                                                                                   
DEBUG RS - h2::client:1286 - client connection bound                                                                                                     
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }                                                                                                                                   
DEBUG RS - h2::proto::connection:138 - Connection; peer=Client
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x0), max_concurrent_streams: 100, initial_window_size: 65536, max_frame_size: 16777215 }
DEBUG RS - h2::codec::framed_write:213 - send frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::codec::framed_read:405 - received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 2147418112 }
DEBUG RS - h2::codec::framed_write:213 - send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 }
DEBUG RS - h2::codec::framed_write:213 - send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) }
DEBUG RS - h2::proto::connection:428 - Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library)
DEBUG RS - h2::codec::framed_read:405 - received frame=Settings { flags: (0x1: ACK) }
DEBUG RS - h2::proto::settings:56 - received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3), flags: (0x1: END_STREAM) }                                         
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) }
DEBUG RS - deno_npm::resolution::graph:927 - @myscope/[email protected] - Resolved lit-html@^3.2.0 to [email protected]                                          
DEBUG RS - deno_npm::resolution::graph:927 - @myscope/[email protected] - Resolved vlq@^2.0.4 to [email protected]                                                    
DEBUG RS - hyper_util::client::legacy::pool:270 - reuse idle connection for ("https", registry.npmjs.org)                                                
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(5), flags: (0x5: END_HEADERS | END_STREAM) }                           
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(5), flags: (0x1: END_STREAM) }                                         
DEBUG RS - deno_npm::resolution::graph:927 - [email protected] - Resolved @types/trusted-types@^2.0.2 to @types/[email protected]                         
DEBUG RS - deno::util::fs:580 - Acquiring file lock at C:\Users\me.me\Downloads\repro\node_modules\.deno\.deno.lock                        
DEBUG RS - deno::util::fs:600 - Acquired file lock at C:\Users\me.me\Downloads\repro\node_modules\.deno\.deno.lock                         
DEBUG RS - hyper_util::client::legacy::pool:270 - reuse idle connection for ("https", some.gitlab.server.com)                                          
DEBUG RS - hyper_util::client::legacy::pool:270 - reuse idle connection for ("https", registry.npmjs.org)
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(3), flags: (0x5: END_HEADERS | END_STREAM) }
DEBUG RS - h2::codec::framed_write:213 - send frame=Headers { stream_id: StreamId(7), flags: (0x5: END_HEADERS | END_STREAM) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }                                                                   
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(7), flags: (0x1: END_STREAM) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) }
DEBUG RS - h2::codec::framed_read:405 - received frame=Data { stream_id: StreamId(3), flags: (0x1: END_STREAM) }
error: Failed caching npm package '@myscope/[email protected]'.                                                                                               
                                                                                                                                                         
Caused by:                                                                                                                                               
    Could not find npm package tarball at: https://some.gitlab.server.com/api/v4/projects/12345/packages/npm/@myscope/packagename/-/@myscope/packagename-14.6.2.tgz

bavoco avatar Sep 28 '24 18:09 bavoco

Hi @bavoco, could you please provide more details on this issue?

Is there any chance you could share the actual scope name and package name? It's somewhat hard to figure out what's going wrong if I'm not sure which parts were edited in the output you provided.

Notice that it did download the dependencies of the package I was trying to install. So although it says it could not find a package tarball, it definitely did find something at the url and it is using the authtoken from the npmrc correcly otherwise it would have had a 404.

So if I get this right - the dependencies of your private package were installed correctly, but the installation of the package itself is erroring when retrieving the tarball?

bartlomieju avatar Oct 01 '24 01:10 bartlomieju

I agree this isn't much to go on, I was hoping using an empty deno cache dir and debug log would be enough information.

I'll try to reproduce this with a public GitLab server later today

bavoco avatar Oct 01 '24 09:10 bavoco

I'm unable to reproduce by publishing a public (and private) package to gitlab.com. Tried same version number, same dependencies, added roughly the same amount of files to it.

However I can consistently reproduce this issue, also on 2.0.0-rc.9, with the business GitLab server. As shown in the log above, deno install correctly finds the dependencies of the package and caches those, but fails caching the package itself. npm install works fine.

Any ideas I can try?

bavoco avatar Oct 01 '24 21:10 bavoco

@bavoco do you have proxies involved in this setup? I'd assume so because of the business GitLab server.

bartlomieju avatar Oct 01 '24 21:10 bartlomieju

I don't know if the requests are being proxied. I tried another GitLab server (on the business network) and deno install works for one of the packages there.

To see if it's caused by the tar, I pointed to the tar file in the package.json and that doesn't install it. Does Deno support specifiying a tar in the package.json?

bavoco avatar Oct 01 '24 21:10 bavoco

For now I can work around the issue with a fetch and @std/tar (very nice that it supports web streams)

bavoco avatar Oct 01 '24 22:10 bavoco

To see if it's caused by the tar, I pointed to the tar file in the package.json and that doesn't install it. Does Deno support specifiying a tar in the package.json?

@bavoco not currently, no.

For now I can work around the issue with a fetch and @std/tar (very nice that it supports web streams)

That's great to hear.

I'm eager to look into this issue as soon as you can provide some more info. I'm gonna remove it from Deno 2.0 milestone for now.

bartlomieju avatar Oct 02 '24 22:10 bartlomieju

I got the same issue. We have a private repo in gitlab and when i try to install with the same .npmrc that works with node i get the same message as op.

We dont have any proxy envolved.

What kind of information do you need? @bartlomieju

chris-clyde avatar Oct 16 '24 09:10 chris-clyde

FYI you can upgrade to canary version to get this fix immediately (deno upgrade canary) or wait until stable Deno v2.0.3 that should be released tonight.

bartlomieju avatar Oct 24 '24 23:10 bartlomieju