Fails when /etc/nix missing

[wmertens]

Had to sudo mkdir /etc/nix before nix-linuxkit-configure would work.

Now it completed but it won't work, the *kit processes go away after a short time.

[zimbatm]

Are you on Mojave @wmertens ? I am seeing a similar issue.

To run the daemon manually use: $(grep nix-linuxkit ~/.cache/nix-linuxkit-builder/gcroot | cut -f 2 -d '>' | cut -f 1 -d '<'), this will give some more logs to see:

+ PATH=/nix/store/acnfbicd84bnya2l2dq868b5l482qihw-coreutils-8.30/bin:/nix/store/2l2801nggm6n8ax4ykbywfa134gj1s9q-openssh-7.9p1/bin:/nix/store/vgsna6b9kyiaw1zj96sds1vhvrh3vly2-gnutar-1.30/bin
+ BOOT_FILES=/nix/store/38nc5jpicqrc00brxg9bwz55q7paswyv-linuxkit-kernel-files
+ VPNKIT_ROOT=/nix/store/kl9xysb9krrirl8jcfbn57vfp0i2fbzy-vpnkit-75434cd
+ HYPERKIT_ROOT=/nix/store/8w5jc0dk66i3ijiq8q249f2r3q5brvl8-hyperkit-6f6edf
+ LINUXKIT_ROOT=/nix/store/0bjpd644ammp5rqq81zcx4wn7r9q3y8x-linuxkit-0.2-bin
+ CONTAINER_IP=192.168.65.2
+ NIX_LINUXKIT_RUNNER=/nix/store/5dicbjhpj4x9da638lkh9fbds8l7jh9g-rust_nix-linuxkit-runner-0.1.0
+ DIR=/Users/zimbatm/.cache/nix-linuxkit-builder/
+ FEATURES=big-parallel
+ SIZE=80
+ CPUS=1
+ MEM=4096
+ VERBOSE=
+ cfg_path=/Users/zimbatm/.cache/nix-linuxkit-builder//configure
+ '[' -f /Users/zimbatm/.cache/nix-linuxkit-builder//configure ']'
+ echo 'Configuration is loaded from /Users/zimbatm/.cache/nix-linuxkit-builder//configure'
Configuration is loaded from /Users/zimbatm/.cache/nix-linuxkit-builder//configure
+ . /Users/zimbatm/.cache/nix-linuxkit-builder//configure
++ FEATURES=kvm,big-parallel
++ SIZE=80
++ MEM=4096
++ VERBOSE=
+ echo

+ echo 'Reconfigure with nix-linuxkit-configure'
Reconfigure with nix-linuxkit-configure
+ echo

+ echo 'Current configuration options:'
Current configuration options:
+ echo FEATURES=kvm,big-parallel
FEATURES=kvm,big-parallel
+ echo SIZE=80
SIZE=80
+ echo MEM=4096
MEM=4096
+ echo VERBOSE=
VERBOSE=
+ exec /nix/store/5dicbjhpj4x9da638lkh9fbds8l7jh9g-rust_nix-linuxkit-runner-0.1.0/bin/nix-linuxkit-runner --linuxkit /nix/store/0bjpd644ammp5rqq81zcx4wn7r9q3y8x-linuxkit-0.2-bin/bin/linuxkit --hyperkit /nix/store/8w5jc0dk66i3ijiq8q249f2r3q5brvl8-hyperkit-6f6edf/bin/hyperkit --vpnkit /nix/store/kl9xysb9krrirl8jcfbn57vfp0i2fbzy-vpnkit-75434cd/bin/vpnkit --ip 192.168.65.2 --disk-size 80 --state-root /Users/zimbatm/.cache/nix-linuxkit-builder/ --cpus 1 --memory 4096 --kernel-files /nix/store/38nc5jpicqrc00brxg9bwz55q7paswyv-linuxkit-kernel-files/nix
Possibly fine error removing the pidfile: Os { code: 2, kind: NotFound, message: "No such file or directory" }
vpnkit: [INFO] Setting handler to ignore all SIGPIPE signals
vpnkit: [INFO] Version %VERSION% from %VCS_COMMIT_ID%
vpnkit: [INFO] System SOMAXCONN is 128
vpnkit: [INFO] Will use a listen backlog of 128
vpnkit: [INFO] Starting port forwarding server on port_control_url:"fd:4" vsock_path:"/Users/zimbatm/.cache/nix-linuxkit-builder/nix-state/connect"
vpnkit: [INFO] Removed connection limit
vpnkit: [WARNING] There is no database: using hardcoded network configuration values
vpnkit: [INFO] No DNS names will map to local IP 192.168.65.1
vpnkit: [INFO] 2 upstream DNS servers are configured
vpnkit: [INFO] Disabling transparent HTTP redirection
vpnkit: [INFO] Updating resolvers to use host resolver
vpnkit: [WARNING] No secure random number generator available
vpnkit: [INFO] Add(3): DNS configuration changed to: use host resolver
vpnkit: [INFO] DNS names [ vpnkit.host ] will map to local IP 192.168.65.1
vpnkit: [INFO] Will use the host's DNS resolver
vpnkit: [INFO] Configuration server_macaddr = f6:16:36:bc:f9:c6; max_connection = None; dns_path = None; dns = ; resolver = Host; domain = None; allowed_bind_addresses = 0.0.0.0; gateway_ip = 192.168.65.1; lowest_ip = 192.168.65.2; highest_ip = 192.168.65.254; extra_dns = ; dhcp_json_path = None; dhcp_configuration = None; mtu = 1500; http_intercept = None; http_intercept_path = None; port_max_idle_time = 30; host_names = vpnkit.host
vpnkit: [INFO] /etc/hosts file has bindings for localhost broadcasthost localhost
vpnkit: [INFO] Vmnet.Server.negotiate: received { magic = VMN3T; version = 22; commit = 6f6edf716b893544c9e0ef3032459180560f0333 }
vpnkit: [INFO] Vmnet.Server.negotiate: received Preferred_ipv4 5d8fe586-1449-4875-acc6-450358db8a5b 192.168.65.2
vpnkit: [INFO] Client requested IP 192.168.65.2
vpnkit: [INFO] Vmnet.Server.negotiate: sending { mtu = 1500; max_packet_size = 1550; client_macaddr = 02:50:00:00:00:01 }
virtio-net-vpnkit: magic=VMN3T version=22 commit=0123456789012345678901234567890123456789??

vpnkit: [INFO] Vmnet.Server.listen: rebinding the primary listen callback
vpnkit: [INFO] Vmnet.Server.listen: starting event loop
vpnkit: [INFO] Connected Ethernet interface f6:16:36:bc:f9:c6
vpnkit: [INFO] Client mac: 02:50:00:00:00:01 server mac: f6:16:36:bc:f9:c6
vpnkit: [INFO] TCP/IP ready
vpnkit: [INFO] TCP/IP stack connected
vpnkit: [INFO] There is no introspection server requested. See the --introspection argument
vpnkit: [INFO] No diagnostics server requested. See the --diagnostics argument
vsock init 3:0 = /Users/zimbatm/.cache/nix-linuxkit-builder/nix-state, guest_cid = 00000003

rdmsr to register 0x34 on vcpu 0

virtio-net-vpnkit: initialising, opts="path=/Users/zimbatm/.cache/nix-linuxkit-builder/nix-state/vpnkit_eth.sock,uuid=5d8fe586-1449-4875-acc6-450358db8a5b,preferred_ipv4=192.168.65.2"

virtio-net-vpnkit: interface will have uuid 5d8fe586-1449-4875-acc6-450358db8a5b

virtio-net-vpnkit: requesting ip 192.168.65.2

virtio-net-vpnkit: Connection established with MAC=02:50:00:00:00:01 and MTU 1500

virtio-9p: initialising path=/Users/zimbatm/.cache/nix-linuxkit-builder/nix-state/vpnkit_port.sock,tag=port

linkname /Users/zimbatm/.cache/nix-linuxkit-builder/nix-state/tty

COM1 connected to /dev/ttys001

COM1 linked to /Users/zimbatm/.cache/nix-linuxkit-builder/nix-state/tty

vpnkit: [ERROR] Vmnet.Server.listen: read EOF so closing connection
vpnkit: [INFO] Vmnet.Server.listen returning Ok()
FATA[0012] Cannot run hyperkit: exit status 2           
linuxkit's ending state: ExitStatus(ExitStatus(256))
Bye!

[wmertens]

Yes, same error, hyperkit failing with exit code 2, which is not documented. Oddly, Docker.App works fine on the same system.

[zimbatm]

All the *kit haven't been upgraded for a while, there might be some mojave-related bugfixes already in upstream.

[wmertens]

oh wow - the hyperkit rev is from 2016. Latest releases:

• hyperkit 06c3cf7ec253534b2d81f61ee3c85c5c9aafa4ee
• vpnkit is a07303459c595d8861e1770b043fd2c252aa0476

[zimbatm]

I created a "staging" branch tracked by hydra: https://hydra.nixos.org/jobset/linux-on-mac-builder/staging

The branch contains upgraded packages, which predictably broke the builds. It would be nice to get some help to fix the builds.

[johnae]

@zimbatm Assume the build failure on staging is similar/same as this: https://github.com/moby/hyperkit/issues/170

Other similar issues: https://github.com/moby/hyperkit/issues?utf8=%E2%9C%93&q=has+incomplete+type

[zimbatm]

Sounds about right. Let's try again once https://github.com/NixOS/nixpkgs/pull/47678 is merged

[takeda]

@zimbatm looks like https://github.com/NixOS/nixpkgs/pull/47678 got merged, what can I do to get linuxkit working again? It also is broken on my Mac (Mojave).

[nicknovitski]

I opened #23 to fix the hyperkit package on the staging branch. I don't know what's up with the other package, it looks like its repository has been removed.

[nicknovitski]

In general the installation script doesn't properly handle a single-user installation. I'm not sure if that's a separate issue from this one.

[nicknovitski]

Staging is now fully green on hydra.

[adrian-gierakowski]

@nicknovitski how could I install the version from staging?

EDIT: so I installed it with nix-env -i /nix/store/zg8h1fv0b3jrgjnmic85lkyard3zvnvx-linuxkit-builder but it still doesn't work.

> launchctl list | grep linuxkit
-	0	org.nix-community.linuxkit-builder

running the executable from the plist directly gives me:

FATA[0000] Cannot run hyperkit: signal: abort trap
linuxkit's ending state: ExitStatus(ExitStatus(256))
Bye!

[nicknovitski]

I've reproduced and fixed the hyperkit error on master. Also I added a link to the hydra jobset from the readme. When it finishes evaluating that change, you can test it out by running nix-env -i with the output store paths on the details tab of the evaluation. That's what I'll be doing. If it seems to work, I'll update the installation instruction.

[adrian-gierakowski]

@nicknovitski thanks! the linuxkit-builder job seems to be stuck in a queue though: https://hydra.nixos.org/eval/1533114#tabs-unfinished

[adrian-gierakowski]

installed the lates build with nix-env -i /nix/store/5bq2klm3386g43xadiaff9jzg75vhrvw-linuxkit-builder but now I get this:

....
hyperkit: stdout: COM1 linked to /Users/adriangierakowski/.cache/nix-linuxkit-builder/nix-state/tty

vpnkit: [ERROR] Vmnet.Server.listen: read EOF so closing connection
vpnkit: [INFO] Vmnet.Server.listen returning Ok()
WARN[0010] hyperkit: failed to read stdout: EOF
WARN[0010] hyperkit: failed to read stderr: EOF
FATA[0010] Cannot run hyperkit: exit status 2
linuxkit's ending state: ExitStatus(ExitStatus(256))
Bye!