Private registry / Configure npm registry for module download

[geglock]

I would like to be able to use a custom, self-hosted npm registry. Especially for resolving company-local private packages based on scopes this would be very helpful. As far as I can see, currently only the npm public registry (https://registry.npmjs.org/) can be used.

Deno should support something similar to npm's configuration:

npm config set registry ...
npm config set @mycompany:registry ...

[jiawei397]

I think it is necessary. I want to pay attention to this progress, because in China, I sometimes download the resources of npm very slowly.

[jimisaacs]

+1 this is a basic requirement for enterprise use

[sithumonline]

Good to have GitLab private package registry support 🚀

[NickTomlin]

It looks like this is currently possible through the DENO_NPM_REGISTRY ENV flag (source)

# execute a remote script ala `npx`
DENO_NPM_REGISTRY=https://my-npm-instance.com deno run npm:@my-internal-scope/my-internal-package
# Start module
cat index.ts

import * as myModule from "npm:@my-internal-scope/my-internal-package"
console.log(myModule)

DENO_NPM_REGISTRY=https://my-npm-instance.com deno run index.ts

It'd be great if this was possible to set via a configuration property in deno.json.

[dsherret]

Sorry, the DENO_NPM_REGISTRY env variable wasn't intended to be stable at this point and was undocumented, but yeah it should work for now. I added a warning on its use in https://github.com/denoland/deno/pull/16953 in future versions, but just beware that it may be changed when we come up with a real solution for this problem (though likely it will still remain... it's just not something that's been discussed yet).

[NickTomlin]

@dsherret thanks for clarifying that and a adding that warning 😄

Is this issue the best place for community feedback to try and help shape that work?

As @jimisaacs mentioned, I think this is crucial in bringing the magic of npm:* to the enterprise space (full disclosure: Jim and I work together at Netflix). Internally, the company has done a lot of investment in scaling, hardening, and auditing our private NPM registry. Being able to leverage that investment directly, instead of having to go through all the same technical and procedural hoops to stand up infrastructure to sit in front of it (e.g. esm.sh), would be huge win.

[dsherret]

We're going to discuss this issue tomorrow in our weekly meeting. For now I'm not adding the warning because the environment variable is used in all our tests in order to use a test npm registry, but consider it unstable.

[ry]

Probably we should just use the same env var as NPM: NPM_CONFIG_REGISTRY

[NickTomlin]

NPM_CONFIG_REGISTRY makes sense to me; I like matching the grain of the existing patterns.

In the meantime, we can dual-write to both variables to make a transition easier.

[dsherret]

I opened https://github.com/denoland/deno/pull/16980 for NPM_CONFIG_REGISTRY support.

We'll look into support for different registries for scopes and credentials in the future.

[lino-levan]

https://github.com/denoland/deno/pull/16980 got merged, should this issue be closed?

[fufuuuu]

So how to set NPM_CONFIG_REGISTRY?

[aapoalas]

Like you would normally set environment variables. Google may help you with this if you do not know how to do it already.

[silverwind]

Ideally deno should support parsing per-scope registry and per-registry auth tokens from the npmrc files and corsponding env vars, so that users can use multiple registries and tokens.

[bartlomieju]

Ideally deno should support parsing per-scope registry and per-registry auth tokens from the npmrc files and corsponding env vars, so that users can use multiple registries and tokens.

It's mentioned in a comment above: https://github.com/denoland/deno/issues/16105#issuecomment-1341330077

[reyawn]

We'll look into support for different registries for scopes and credentials in the future.

Is there any update on this @dsherret? This is a big blocker for many use cases.

[dsherret]

The NPM_CONFIG_REGISTRY environment variable landed in 1.29, but we haven't added support for different registries per scope yet.

[marziply]

Where I work, there are some Node microservices that could benefit from migrating to Deno. However, these microservices depend on private Gitlab NPM packages. This feature is blocking the upgrade to Deno because many of these private packages are critical to these services and cannot run without them.

The work so far on this has been excellent and I've not reported any other issues. I will just have to wait until Deno supports private packages before upgrading.

[marcushultman]

Implementing this as a Configuration file option ("npmRegistry") in https://github.com/denoland/deno/pull/19317

[esnho]

Hi, how is possible to setup a private registry with this environment variable?

I'm using gemfury as registry, and I've tried below command without success:

DENO_NPM_REGISTRY=https://npm-proxy.fury.io/myorg/:_authToken=${GEMFURY_DEPLOY_TOKEN} deno run myregistry.ts

I get this error:

error: npm package '@myorg/mypackage' does not exist.
    at file:///Users/user/temp/deno-test/myregistry.ts:1:29

I usually use gemfury with yarn on node and it works nicely, below my .npmrc file (note that has been redacted)

@myorg:registry=https://npm-proxy.fury.io/myorg/
//npm-proxy.fury.io/myorg/:_authToken=${GEMFURY_DEPLOY_TOKEN}

and finally here is myregistry.ts

import { MyModyle } from "npm:@myorg/mypackage";
console.log(MyModule)

[marcushultman]

@esnho The env was changed to NPM_CONFIG_REGISTRY.

[felipemullen]

Setting NPM_CONFIG_REGISTRY almost does the job. In my particular case, there are scoped packages sitting in an artifactory registry that does not proxy npm directly. This results in two scenarios:

NPM_CONFIG_REGISTRY is set to my artifactory registry

• @myorg/my-package is resolved correctly ✔️
• global npm packages such as express are not ❌

NPM_CONFIG_REGISTRY is not set

• @myorg/my-package is not resolved correctly ❌
• global npm packages such as express are fine ✔️

Is there a way I could have two registries set? I tried using spaces and semicolons ; in the string, no success. My artifactory uses an auth user with token, which is defined in .npmrc

[bhb603]

@felipemullen I have the same issue as you...

One workaround I found was to use a local proxy like Verdaccio to proxy the various registries you need.

Caveat: I don't have prior experience with Verdaccio, or setting up my own local npm proxy; I just discovered it when looking for solutions to this multi-registry limitation in Deno 😄

Anyway, I have Verdaccio configured with the following uplinks and package resolutions:

uplinks:
  npmjs:
    url: https://registry.npmjs.org/
  github_private:
    url: https://npm.pkg.github.com/
    auth:
      type: bearer
      token_env: GITHUB_TOKEN
packages:
  '@myorg/*':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: github_private

  '**':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: npmjs

So all @myorg scoped packages will pull from Github private registry, and the rest will pull from npm.

Then run verdaccio locally, supplying any necessary auth tokens:

GITHUB_TOKEN=*** verdaccio

Then set the Verdaccio server (localhost:4873 by default) as NPM_CONFIG_REGISTRY and use Deno to cache your dependencies:

NPM_CONFIG_REGISTRY=http://localhost:4873 deno cache ./my_deno_module.ts

[andykais]

just throwing another monkey wrench into this discussion. I have a use case that involves using npm git dependencies. My company has private npm packages that we simply create git tags for and host on github. These are then defined in package.json like so:

{
  "dependencies": {
    "mypackage": "git+ssh://[email protected]:myorganization/mypackage.git#3.1.1"
  }
}

The packages themselves contain native code, and use the npm script "install": "node-gyp rebuild" to compile them. This works well for us, the only requirement is that we have a relevant git ssh key added wherever the package needs to be installed. In deno however, this does not appear to work:

import * as mypackage from 'npm:git+ssh://[email protected]:myorganization/mypackage.git#3.1.1'

This results in the following error:

error: Invalid package specifier 'npm:git+ssh://[email protected]:myorganization/mypackage.git#3.1.1'. Did you mean to write 'npm:git+ssh:@github.com:myorganization/mypackage.git#3.1.1//git'?

Is the git ssh reference part of the npm spec supported in deno's npm specifiers?

[felipemullen]

Thanks @bhb603, that was a great tip. I was able to get set up using docker-compose to support a self-hosted windmill.dev instance.

For anyone else struggling with the same, here are some notes:

docker-compose.yaml

services:
  verdaccio:
    container_name: verdaccio
    image: verdaccio/verdaccio
    environment:
      - VERDACCIO_PORT=4873
    ports:
      - 4873:4873
    expose:
      - 4873
    volumes:
      - './path/to/local/config:/verdaccio/conf'

windmill_worker:
    ...
    environment:
      - NPM_CONFIG_REGISTRY=http://verdaccio:4873

my/local/verddaccio/config/config.yaml

uplinks:
  npmjs:
    cache: false
    url: https://registry.npmjs.org/
  artifactory:
    cache: false
    url: https://mycorp.artifactory.corp.com/api/npm/my-packages/
    headers:
      Authorization: bearer ABC123ZYX098===AABBCOLLISION

packages:
  '@mycorp/*':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: artifactory

  '**':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: npmjs

Docs references:

• private npm registries for windmill - https://www.windmill.dev/docs/advanced/imports#private-npm-registry
• docker networking reference - https://docs.docker.com/compose/networking/
• verdaccio docker-compose docs - https://verdaccio.org/docs/docker#using-docker-compose

[dehrhard]

I second this, I want this so bad. We wouldn't even need scoped support for other registries. Since github is proxying the requests through to npmjs we would only need support for auth with github.

[nitehogg]

Is this currently being worked on, or has a decision been made towards the implementation yet?

[rattrayalex]

As a package maintainer, I'd like to be able to test packages I'm developing on Deno before publishing (as well as get feedback from external users before a given package or package version hits a registry).

The following (or some equivalent) would all be quite useful for being able to offer a great experience to Deno users of my package:

import mypackage from 'npm:./some-bundle.tar.gz'
import mypackage from 'npm:./some-dir'
import mypackage from 'npm:github.com/my-org/my-repo#v0.2.1-beta'

[bartlomieju]

Update: we are now working on supporting .npmrc file that will allow to use different registries for different scopes.

[bartlomieju]

Update: We're gonna land first pass in #23560 really soon.

The first pass will support .npmrc located next to package.json or deno.json - so it will work with Node.js first projects as well as Deno first projects. Please let me know if you rely on .npmrc in your user directory or global directory (https://docs.npmjs.com/cli/v10/configuring-npm/npmrc#files).

Only _authToken config will be supported in the first pass - please let me know by commenting if you are heavy users of other options so we can prioritize accordingly.

Also the first pass will only support deno run / deno test subcommands - the support for deno compile will come afterwards - please also comment if you need it.

Thanks