Not working with NPM?

[arctic-ice-cool]

We have multiple private repositories across both Composer and NPM. I set this up and it pretty much instantly started working with Composer - however I can't get it to work with NPM - no matter which format I place within the dependency in package.json (HTTP, git, git+ssh etc)

Is this possible with this package?

[joshbradfield-kiwirail]

I also have not got this to work with NPM.

When I test with git git ls-remote git@github.[redacted]/[redacted].git i get :

Warning: Permanently added the ECDSA host key for IP address '140.82.112.3' to the list of known hosts.
b6bd4fedc4[28](https://github.com/[redacted]/[redacted]/runs/8050362856?check_suite_focus=true#step:6:29)007d898b5f5d5b3f5f502bf7d180	HEAD
b6bd4fedc428007d898b5f5d5b3f5f502bf7d180	refs/heads/main
2a525e53557497744e6774604aae71b1a87775a7	refs/tags/v0.0.6
b6bd4fedc428007d898b5f5d5b3f5f502bf7d180	refs/tags/v0.0.6^{}

But when I try to use NPM on the same repository npm view [email protected]:[redacted]/[redacted].git:

npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects ls-remote ssh://[email protected]/[redacted]/dd-[redacted].git
npm ERR! ssh: Could not resolve hostname key-7229bcc530b8030b34688f3e951db9e6114e46db8832365ede2298bfbbd153fe.github.com: Name or service not known
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.

npm ERR! A complete log of this run can be found in:
npm ERR!     /github/home/.npm/_logs/2022-08-27T12_34_11_502Z-debug.log
Error: Process completed with exit code 128.

There doesn't seem to be much online, except one person posting that they think NPM and ssh-agent don't play nicely together https://github.com/npm/cli/issues/2614 (though rejected out of hand by the NPM team, so maybe its a conflict somewhere?)

[joshbradfield-kiwirail]

I tested two different environments, seems to work as expected under the windows env but not ubuntu

• 'windows-latest' ✔
  • keys added and mapped ✔
  • git ls-remote ✔
  • npm view ✔
• 'ubuntu-latest'❌
  • keys added and mapped ✔
  • git ls-remote ✔
  • npm view ❌

[arctic-ice-cool]

I have been trying all sorts for days to get this working, it just doesn’t work with NPM 😞

[joshbradfield-kiwirail]

I have been trying all sorts for days to get this working, it just doesn’t work with NPM 😞

What environment are you use? ubuntu-latest or windows-latest?

[arctic-ice-cool]

What environment are you use? ubuntu-latest or windows-latest?

ubuntu-22.04

[arctic-ice-cool]

It's like NPM just isn't using the SSH-agent, from what I can gather. NPM should do though, as it works on Windows as you say, just not in Github Actions on Ubuntu 22.04...

[mpdude]

How do you specify the package source in your package.json?

[arctic-ice-cool]

How do you specify the package source in your package.json?

Literally every possible way to reference a package source I can think of. Including changing [email protected] to <username>@github.com!

Current method that I gave up on:

git+ssh://[email protected]/organization/package-name.git

[mpdude]

Can you set the environment variable as described here?

https://askubuntu.com/a/620985

Maybe that can give us a little bit more insight what’s going on under the hood?

[mpdude]

For the record, I was able to clone a private repo on my personal machine with a package URL like git+ssh://[email protected]/../....git.

When I unset the SSH_AUTH_SOCK environment variable, which is needed to find the local SSH agent, this no longer works. So, npm must be using the agent under the hood, although I do not know whether it shells out to a regular git and/or ssh command under the hood, or what part of those it implements natively.

$ npm -v
8.15.0

[arctic-ice-cool]

It just gives this output:

npm ERR! code E401
npm ERR! Incorrect or missing password.
npm ERR! If you were trying to login, change your password, create an
npm ERR! authentication token or enable two-factor authentication then
npm ERR! that means you likely typed your password in incorrectly.
npm ERR! Please try again, or recover your password at:
npm ERR!     https://www.npmjs.com/forgot
npm ERR! 
npm ERR! If you were doing some other operation then your saved credentials are
npm ERR! probably out of date. To correct this please try logging in again with:
npm ERR!     npm login

But this isn't an npm auth failure, it's a failure to access a private GitHub repo - but composer authenticates fine using this SSH Agent - just not NPM 🤔

[skinofstars]

Hey, just throwing a +1 on this. Still having the issue today. Has anyone found a workaround?

[arctic-ice-cool]

Still having this issue too 🫤

[mpdude]

When you set the GIT_SSH_COMMAND="ssh -vvv" env var while running npm, does that produce any useful debugging output?

[skinofstars]

Here's the jist of it

jobs:
  test:
    runs-on: ubuntu-latest

    container:
      image: node:14.21

    services:
      postgres:
        image: postgres
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v3
      - uses: webfactory/[email protected]
        with:
          ssh-private-key: ${{ secrets.PRIVATE_KEY }}

      - name: install
        run: npm ci
        env:
          GIT_SSH_COMMAND: 'ssh -vvv'

      - run: npm run build --if-present

      - name: tests
        run: >
          PG_CONNECTION_STRING_=postgres://postgres:postgres@postgres:5432/postgres
          npm test

Output from the action (I've replaced some ORG/REPO/IDs)

Run webfactory/[email protected]
  with:
    ssh-private-key: ***
  
    log-public-key: true
/usr/bin/docker exec  fc3fe5fed2bbfe503480a81edec36c165edd7d5247806ef7fc57b6b262856c8b sh -c "cat /etc/*release | grep ^ID"
Adding GitHub.com keys to /root/.ssh/known_hosts
Starting ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-mM9c0Z3SXa4K/agent.125
SSH_AGENT_PID=126
Adding private key(s) to agent
Identity added: (stdin) ([email protected]:<ORG>/<REPO>.git)
Key(s) added:
4096 SHA256:<XXXXXX>/uEmn8 [email protected]:<ORG>/<REPO>.git (RSA)
Configuring deployment key(s)
Added deploy-key mapping: Use identity '/root/.ssh/key-<XXXXXX>' for GitHub repository <ORG>/<REPO>

And output from the install task

Run npm ci
  npm ci
  shell: sh -e {0}
  env:
    SSH_AUTH_SOCK: /tmp/ssh-mM9c0Z3SXa4K/agent.125
    SSH_AGENT_PID: 126
    GIT_SSH_COMMAND: ssh -vvv
npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t ssh://[email protected]/<ORG>/<REPO>.git
npm ERR! 
npm ERR! No user exists for uid 1001
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR! 
npm ERR! exited with error code: 128

npm ERR! A complete log of this run can be found in:
npm ERR!     /github/home/.npm/_logs/2022-11-25T09_42_28_897Z-debug.log
Error: Process completed with exit code 1.

[mpdude]

Can you execute /usr/bin/git ls-remote -h -t ssh://[email protected]/<ORG>/<REPO>.git as a stand-alone command in this workflow?

Does it do anything reasonable, or fail with authentication as well?

(Maybe add GIT_SSH here as well?)

[mpdude]

Oh wait, what does this do?

container:
      image: node:14.21

Is all this, including the SSH agent, running in a Docker container?

Is that the same situation for everyone else here affected by the bug?

[skinofstars]

Ok, so two additional run experiments.

- run: git clone [email protected]:<ORG>/<REPO>.git successfully clones

- run: /usr/bin/git ls-remote -h -t ssh://[email protected]/<ORG>/<REPO>.git successfully returns refs for active branches

So this does seem to be an NPM issue.

Yeah, we're in a container, which you've gotta do if you need other services to test against e.g. a database.

I ran my steps outside of the container and it seems to have worked fine, so looks like this is a part of the issue.

[mpdude]

What about this?

No user exists for uid 1001

Maybe git or ssh have trouble figuring out which/where the current user's home directory is, which might be necessary to locate config files?

[felixlut]

What about this?

No user exists for uid 1001

Maybe git or ssh have trouble figuring out which/where the current user's home directory is, which might be necessary to locate config files?

TLDR: Running a container as root and using this action does not work properly currently

I can confirm that this is indeed the reason. When running for example the regular node image, you are running as root. This makes this action install the SSH key in /root/.ssh/:

I haven't dug too deep into it, but I suspect that npm tried to find the SSH keys under some other directory (probably derived from the No user exists for uid 1001 message, which fails the npm command). When trying to use pnpm, the installation process does work, so I think that one does a more exhaustive search of the machine's SSH keys.

Creating a new container image from node where a new user (sonarqube-runner in my case) is created and used, this action does install the SSH keys in a more "normal" place:

This worked. I'm not digging any deeper.