runtime icon indicating copy to clipboard operation
runtime copied to clipboard

Published 20 hours ago • verified publisher icon clearcontainers

Fail to start "ros" image with CC

Open thiagomacieira opened this issue 6 years ago • 12 comments

When trying to run the "ros" image with CC, I get a cryptic error message:

$ docker run -it ros     
cat: -: No such file or directory

The same image works fine with runc. Clear Containers appears to be working fine:

$ docker run -it busybox cat /proc/version
Linux version 4.9.47-77.5.container (abuild@) (gcc version 7.2.1 20170901 [gcc-7-branch revision 251580] (SUSE Linux) ) #1 SMP Fri Sep 15 18:23:40 UTC 2017

thiagomacieira avatar Sep 18 '17 20:09 thiagomacieira

@thiagomacieira I can't reproduce it on my setup:

$ docker run -ti --runtime=cc ros
root@6eee79f5ed32:/# uname -r
4.9.47-77.1.container
$ cc-runtime cc-env
root@6eee79f5ed32:/# exit
[Meta]
  Version = "1.0.3"

[Runtime]
  [Runtime.Version]
    Semver = "3.0.0-beta.2"
    Commit = "c9ccbb7ff61b3546506c975dcef9421f9b270366"
    OCI = "1.0.0-rc5"
  [Runtime.Config]
    GlobalLogPath = ""
    [Runtime.Config.Location]
      Path = "/etc/clear-containers/configuration.toml"
      Resolved = "/etc/clear-containers/configuration.toml"

[Hypervisor]
  MachineType = "pc"
  Version = "QEMU emulator version 2.7.0, Copyright (c) 2003-2016 Fabrice Bellard and the QEMU Project developers"
  [Hypervisor.Location]
    Path = "/usr/bin/qemu-lite-system-x86_64"
    Resolved = "/usr/bin/qemu-lite-system-x86_64"

[Image]
  Path = "/usr/share/clear-containers/clear-containers.img"
  Resolved = "/usr/share/clear-containers/clear-17270-containers.img"

[Kernel]
  Parameters = ""
  [Kernel.Location]
    Path = "/usr/share/clear-containers/vmlinuz.container"
    Resolved = "/usr/share/clear-containers/vmlinuz-4.9.47-77.1.container"

[Proxy]
  Type = "ccProxy"
  Version = "Version: 3.0.0-beta.2-18-g0d66493-dirty"
  URL = "unix:///var/run/clear-containers/proxy.sock"

[Shim]
  Type = "ccShim"
  Version = "shim version: 3.0.0-alpha.4 (commit: 0)"
  [Shim.Location]
    Path = "/usr/libexec/clear-containers/cc-shim"
    Resolved = "/usr/libexec/clear-containers/cc-shim"

[Agent]
  Type = "hyperstart"
  Version = "<<unknown>>"
  [Agent.PauseBin]
    Path = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"
    Resolved = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"

[Host]
  Kernel = "4.12.5-300.fc26.x86_64"
  CCCapable = true
  [Host.Distro]
    Name = "Fedora"
    Version = "26"
  [Host.CPU]
    Vendor = "GenuineIntel"
    Model = "Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz"

Could you please dump your cc-runtime cc-env output ?

sameo avatar Sep 18 '17 22:09 sameo 

[Meta]
  Version = "1.0.2"

[Runtime]
  [Runtime.Version]
    Semver = "3.0.0-beta.1"
    Commit = "3d44244608b5cbe9e1295520aa9f54a494954b23"
    OCI = "1.0.0-rc5"
  [Runtime.Config]
    GlobalLogPath = ""
    [Runtime.Config.Location]
      Path = "/etc/clear-containers/configuration.toml"
      Resolved = "/etc/clear-containers/configuration.toml"

[Hypervisor]
  MachineType = "pc"
  [Hypervisor.Location]
    Path = "/usr/bin/qemu-system-x86_64"
    Resolved = "/usr/bin/qemu-system-x86_64"

[Image]
  Path = "/usr/share/clear-containers/clear-containers.img"
  Resolved = "/usr/share/clear-containers/clear-17270-containers.img"

[Kernel]
  Parameters = ""
  [Kernel.Location]
    Path = "/usr/share/clear-containers/vmlinuz.container"
    Resolved = "/usr/share/clear-containers/vmlinuz-4.9.47-77.5.container"

[Proxy]
  Type = "ccProxy"
  Version = "<<unknown>>"
  URL = "unix:///var/run/clear-containers/proxy.sock"

[Shim]
  Type = "ccShim"
  Version = "shim version: 3.0.0-alpha.4 (commit: 0)"
  [Shim.Location]
    Path = "/usr/lib64/clear-containers/cc-shim"
    Resolved = "/usr/lib64/clear-containers/cc-shim"

[Agent]
  Type = "hyperstart"
  Version = "<<unknown>>"
  [Agent.PauseBin]
    Path = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"
    Resolved = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"

[Host]
  Kernel = "4.12.11-1-default"
  CCCapable = false
  [Host.Distro]
    Name = "openSUSE Tumbleweed"
    Version = "20170913"
  [Host.CPU]
    Vendor = "GenuineIntel"
    Model = "Intel(R) Core(TM) i7-6560U CPU @ 2.20GHz"

thiagomacieira avatar Sep 18 '17 22:09 thiagomacieira

Would you mind upgrading to beta.2 and verifying that this still fails ?

sameo avatar Sep 18 '17 23:09 sameo

Same error:

[Meta]
  Version = "1.0.2"

[Runtime]
  [Runtime.Version]
    Semver = "3.0.0-beta.2"
    Commit = "06023ea"
    OCI = "1.0.0-rc5"
  [Runtime.Config]
    GlobalLogPath = ""
    [Runtime.Config.Location]
      Path = "/etc/clear-containers/configuration.toml"
      Resolved = "/etc/clear-containers/configuration.toml"

[Hypervisor]
  MachineType = "pc"
  [Hypervisor.Location]
    Path = "/usr/bin/qemu-system-x86_64"
    Resolved = "/usr/bin/qemu-system-x86_64"

[Image]
  Path = "/usr/share/clear-containers/clear-containers.img"
  Resolved = "/usr/share/clear-containers/clear-17270-containers.img"

[Kernel]
  Parameters = ""
  [Kernel.Location]
    Path = "/usr/share/clear-containers/vmlinuz.container"
    Resolved = "/usr/share/clear-containers/vmlinuz-4.9.47-77.5.container"

[Proxy]
  Type = "ccProxy"
  Version = "<<unknown>>"
  URL = "unix:///var/run/clear-containers/proxy.sock"

[Shim]
  Type = "ccShim"
  Version = "shim version: 3.0.0-alpha.4 (commit: 1f545df)"
  [Shim.Location]
    Path = "/usr/lib64/clear-containers/cc-shim"
    Resolved = "/usr/lib64/clear-containers/cc-shim"

[Agent]
  Type = "hyperstart"
  Version = "<<unknown>>"
  [Agent.PauseBin]
    Path = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"
    Resolved = "/var/lib/clear-containers/runtime/bundles/pause_bundle/bin/pause"

[Host]
  Kernel = "4.12.11-1-default"
  CCCapable = false
  [Host.Distro]
    Name = "openSUSE Tumbleweed"
    Version = "20170913"
  [Host.CPU]
    Vendor = "GenuineIntel"
    Model = "Intel(R) Core(TM) i7-6560U CPU @ 2.20GHz"

thiagomacieira avatar Sep 19 '17 00:09 thiagomacieira

If I edit /etc/clear-containers/configuration.toml and change the hypervisor to qemu-lite-system-x86_64, it runs.

thiagomacieira avatar Sep 19 '17 00:09 thiagomacieira

To confirm, using the system qemu also fails for me, on ubuntu server 16.04. I enabled the runtime logs, and I see in there:

2017-09-19 10:55:20.023922685 +0100 BST:25281:cc-runtime:error:Unable to launch qemu: exit status 1
2017-09-19 10:55:20.025538082 +0100 BST:25281:cc-runtime:error:qemu-system-x86_64: Property '.nvdimm' not found

The system qemu version installed is: QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.8), Copyright (c) 2003-2008 Fabrice Bellard

Having a quick stare at the QEMU git, it looks like initial nvdimm support went in almost exactly at the time v2.5.0 was released (early Dec. 2015). I guess there was a later addition we rely upon. Conclusion: that nearly 2 year old QEMU is probably too old to have all the features we require. I don't think we actually have a minimum QEMU version listed.

grahamwhaley avatar Sep 19 '17 10:09 grahamwhaley

An addition... I was going to ask how you got the system default QEMU path in your config file @thiagomacieira , but having a peek here: https://github.com/clearcontainers/runtime/blob/master/Makefile#L87 I wondered if you were on Centos or RHEL - but I see above you are on openSUSE. I'm not up on the QEMU packaging names. /cc @gorozco1 @jcvenegas for any input and thoughts.

grahamwhaley avatar Sep 19 '17 10:09 grahamwhaley

@grahamwhaley, that came from the binary packages provided via OBS: https://build.opensuse.org/project/show/home:clearcontainers:clear-containers-3-staging

The .spec file doesn't appear to be doing any shenanigans to the config file, so that came from source. From your link it looks like it should be the lite version, but wasn't. I don't know why.

thiagomacieira avatar Sep 19 '17 16:09 thiagomacieira

@thiagomacieira we are currently building packages in staging and release... and it will take sometime to update.. it might be that. the service in OBS is building the releases.

gorozco1 avatar Sep 20 '17 18:09 gorozco1

@thiagomacieira what is the output of this code in your Tumbleweed:

distro=$(for file in /etc/os-release /usr/lib/os-release; do if [ -e $file ]; then grep ^ID= $file|cut -d= -f2-|tr -d '"';  break; fi done)

echo $distro

gorozco1 avatar Sep 20 '17 19:09 gorozco1 

sh-4.4$ for file in /etc/os-release /usr/lib/os-release; do if [ -e $file ]; then grep ^ID= $file|cut -d= -f2-|tr -d '"';  break; fi; done
opensuse

thiagomacieira avatar Sep 20 '17 20:09 thiagomacieira

@thiagomacieira this is due a missing requirement in OpenSuSE.

There is PR in the work to fix the detection of the distro and apply the proper qemu-lite binary.

https://github.com/clearcontainers/packaging/pull/116

gorozco1 avatar Sep 20 '17 21:09 gorozco1