nix-docker icon indicating copy to clipboard operation
nix-docker copied to clipboard

cannot build on 'ssh://root@nix-docker'

Open o1lo01ol1o opened this issue 5 years ago • 7 comments

I'm following the macOS-nix-docker guide here. I installed nix-darwin with

nix-build https://github.com/LnL7/nix-darwin/archive/master.tar.gz -A installer
./result/bin/darwin-installer

and I've set

nix.distributedBuilds = true;
nix.buildMachines = [ {
  hostName = "nix-docker";
  sshUser = "root";
  sshKey = "/etc/nix/docker_rsa";
  systems = [ "x86_64-linux" ];
  maxJobs = 2;
} ];

and

services.nix-daemon.enable = true;

darwin-rebuild switch:

darwin-rebuild switch
building the system configuration...
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
user defaults...
setting up user launchd services...
setting up ~/Applications...
setting up /etc...
warning: not linking environment.etc."nix/nix.conf" because /etc/nix/nix.conf exists, skipping...
system defaults...
setting up launchd services...

( The warning is because I have a vanilla install of nix as well and hence a nix.conf; maybe this would lead to the failure below?)

I can ssh as root to the docker builder, but running the test build fails with

$ nix-build -E 'with import <nixpkgs> { system = "x86_64-linux"; }; hello.overrideAttrs (drv: { rebuild = builtins.currentTime; })'
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
these derivations will be built:
  /nix/store/8f0dgnis487dsr6xz0d9jyqrhqqvfshb-hello-2.10.drv
cannot build on 'ssh://root@nix-docker': cannot connect to 'root@nix-docker': ssh: Could not resolve hostname nix-docker: nodename nor servname provided, or not known
error: a 'x86_64-linux' is required to build '/nix/store/8f0dgnis487dsr6xz0d9jyqrhqqvfshb-hello-2.10.drv', but I am a 'x86_64-darwin'

Is this an issue with the nix-daemon? How can I get it to talk to the docker container?

o1lo01ol1o avatar May 23 '19 21:05 o1lo01ol1o

This is an issue with the setup not the container. It's trying to resolve nix-docker as a hostname so I'd look at /var/root/.ssh/configt first.

LnL7 avatar May 24 '19 17:05 LnL7

/var/root/.ssh/config is created according to the guide with the following contents:

Host nix-docker
  User root
  HostName 127.0.0.1
  Port 3022
  IdentityFile /etc/nix/docker_rsa

o1lo01ol1o avatar May 24 '19 17:05 o1lo01ol1o

Does it have the correct permissions (600) and ownership?

LnL7 avatar May 24 '19 18:05 LnL7

I believe so:

$ sudo ls -l /var/root/.ssh/
total 16
-rw-------  1 root  wheel   95 May 23 15:29 config
-rw-r--r--  1 root  wheel  398 May 23 15:30 known_hosts
$ ls -l /etc/nix/
total 16
-rw-------  1 root  wheel  1675 May 23 15:26 docker_rsa

o1lo01ol1o avatar May 24 '19 18:05 o1lo01ol1o

I stumbled upon this issue also, a year later.

I assume that there is a domain name lookup during the test which does not respect the setting in ~/.ssh/config. I solved it by abusing the fact that macOS does not try to resolve domain names in the .local tld since they are reserved for Apple's Bonjour zeroconf networking (https://en.wikipedia.org/wiki/.local) and therefor naming the container nix-docker.local instead of nix-docker So the section in ~/.ssh/config looks like:

Host nix-docker.local
  User root
  HostName 127.0.0.1
  Port 3022
  IdentityFile /etc/nix/docker_rsa

/etc/nix/machines is changed likewise. The remote builder is run as docker run --restart always --name nix-docker.local -d -p 3022:22 lnl7/nix:ssh

In hindsight, the real solution is probably to add the line 127.0.0.1 nix-docker to the hosts file (/etc/hosts).

bxrt avatar Aug 16 '20 10:08 bxrt

I confirmed that @bxrt method of adding to /etc/hosts works for me

takedawilliam avatar Jan 28 '21 11:01 takedawilliam

adding the entry to /var/root/.ssh/config works for me, I guess it's because nix-daemon is running as root.

yihuang avatar Feb 21 '22 10:02 yihuang