qus icon indicating copy to clipboard operation
qus copied to clipboard

When running x86 docker image on an arm64 host, the PS command cannot be used in the container

Open HelloGril opened this issue 3 years ago • 12 comments

1.HOST

# uname -m
aarch64
# lscpu
架构:                           aarch64
CPU 运行模式:                   64-bit
字节序:                         Little Endian
CPU:                             8
在线 CPU 列表:                  0-7
每个核的线程数:                 1
每个座的核数:                   1
座:                             8
NUMA 节点:                      1
厂商 ID:                        
型号:                           0
型号名称:                       
步进:                           0x1
CPU 最大 MHz:                   2600.0000
CPU 最小 MHz:                   2600.0000
BogoMIPS:                       200.00
L1d 缓存:                       512 KiB
L1i 缓存:                       512 KiB
L2 缓存:                        4 MiB
L3 缓存:                        256 MiB
NUMA 节点0 CPU:                 0-7
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
标记:fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm

2. container

# uname -m
x86_64
# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             8
NUMA node(s):          1
Model:                 0
Model name:            ARMv8 CPU
CPU max MHz:           2600.0000
CPU min MHz:           2600.0000
BogoMIPS:              200.00
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              32768K
NUMA node0 CPU(s):     0-7
Flags:  fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm

3. Execute PS command in container

# ps
/usr/bin/ps: error while loading shared libraries: libsystemd.so.0: ELF load command alignment not page-aligned
# ss
Netid State      Recv-Q Send-Q                                  Local Address:Port                                                   Peer Address:Port                
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported

HelloGril avatar Dec 22 '21 09:12 HelloGril

I am not sure whether these are the problems but the following two lines look suspicious to me:

# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit                                << I'd expect to see "32-bit, 64-bit" instead
...
Model name:            ARMv8 CPU                        << WHY ARMv8 for the x86_64 container ?!

martin-g avatar Dec 22 '21 09:12 martin-g

I do not kown "WHY ARMv8 for the x86_64 container and Why CPU is 32-bit",I'll send out the command to execute image to help analyze it

# docker run -dt --sysctl net.core.somaxconn=1024 --name "${container_name}" --rm -v /usr/bin/qemu-x86_64-static:/usr/bin/qemu-x86_64-static --platform=amd64  --privileged="${privileged}" ${mounts} ${ports_params} \
        "${hub_address}aciot/aclite:${aclite_tag}

HelloGril avatar Dec 22 '21 10:12 HelloGril

When using QEMU user mode, certain instructions corresponding to a foreign architecture are translated, however, there is no fully isolated virtual machine. Therefore, some data, such as the CPU Model name is showing the original values, not "translating" them. Anyway, the CPU op-mode feels wrong. That might be due to lack of some x86_64 to aarch64 translation.

Nonetheless, it would helpful if you provided further details in order to try reproducing the issue:

  • What is the CPU model on the host?
  • Which is the OS on the host?
  • Does ps work on the host?
  • How did you install the /usr/bin/qemu-x86_64-static binary that you are sharing with the container?
  • Which is the base image of the container you are using?

umarcor avatar Dec 22 '21 10:12 umarcor

  • What is the CPU model on the host? Kunpeng-920
  • Which is the OS on the host? can't find
  • Does ps work on the host? yes
  • How did you install the /usr/bin/qemu-x86_64-static binary that you are sharing with the container? github
  • Which is the base image of the container you are using? In the product package provided by a service provider

HelloGril avatar Dec 22 '21 11:12 HelloGril

* Which is the OS on the host?
  can't find

Can you get the kernel version?

* How did you install the /usr/bin/qemu-x86_64-static binary that you are sharing with the container?
  github

From this repository or from some other repository? Which version of QEMU is it?

* Which is the base image of the container you are using?
  In the product package provided by a service provider

Can you try some "official" image, such as debian:bullseye-slim?

umarcor avatar Dec 22 '21 11:12 umarcor

  • Which is the OS on the host? can't find

cat /etc/*release*

martin-g avatar Dec 22 '21 11:12 martin-g

@umarcor The discussion first started at https://github.com/multiarch/qemu-user-static/issues/161. There I recommended dbhi/qus because @HelloGril wanted to emulate x86_64 on aarch64 host.

martin-g avatar Dec 22 '21 11:12 martin-g

  • Which is the OS on the host?
Kylin Linux Advanced Server release V10 (Sword)
DISTRIB_ID=Kylin
DISTRIB_RELEASE=V10
DISTRIB_CODENAME=juniper
DISTRIB_DESCRIPTION="Kylin V10"
DISTRIB_KYLIN_RELEASE=V10
DISTRIB_VERSION_TYPE=enterprise
DISTRIB_VERSION_MODE=normal
NAME="Kylin Linux Advanced Server"
VERSION="V10 (Sword)"
ID="kylin"
VERSION_ID="V10"
PRETTY_NAME="Kylin Linux Advanced Server V10 (Sword)"
ANSI_COLOR="0;31"

Kylin Linux Advanced Server release V10 (Sword)
cpe:/o:kylin:kylin:10:ga:server
  • Can you try some "official" image, such as debian:bullseye-slim? i will try

  • Which version of QEMU is it? Dwload address:https://objects.githubusercontent.com/github-production-release-asset-2e65be/47342812/2e141383-cb90-4d14-9952-9dfa7fc0fa0f?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211216%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211216T035426Z&X-Amz-Expires=300&X-Amz-Signature=27b7f1477b515916d7c94cdda29e3046a408b984187c9fdcb05a4677f34a66a1&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=47342812&response-content-disposition=attachment%3B%20filename%3Dqemu-x86_64-static.tar.gz&response-content-type=application%2Foctet-stream

HelloGril avatar Dec 22 '21 12:12 HelloGril

1.HOST

# uname -m
aarch64
# lscpu
架构:                           aarch64
CPU 运行模式:                   64-bit
字节序:                         Little Endian
CPU:                             8
在线 CPU 列表:                  0-7
每个核的线程数:                 1
每个座的核数:                   1
座:                             8
NUMA 节点:                      1
厂商 ID:                        
型号:                           0
型号名称:                       
步进:                           0x1
CPU 最大 MHz:                   2600.0000
CPU 最小 MHz:                   2600.0000
BogoMIPS:                       200.00
L1d 缓存:                       512 KiB
L1i 缓存:                       512 KiB
L2 缓存:                        4 MiB
L3 缓存:                        256 MiB
NUMA 节点0 CPU:                 0-7
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
标记:fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm

2. container

# uname -m
x86_64
# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             8
NUMA node(s):          1
Model:                 0
Model name:            ARMv8 CPU
CPU max MHz:           2600.0000
CPU min MHz:           2600.0000
BogoMIPS:              200.00
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              32768K
NUMA node0 CPU(s):     0-7
Flags:  fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm

3. Execute PS command in container

# ps
/usr/bin/ps: error while loading shared libraries: libsystemd.so.0: ELF load command alignment not page-aligned
# ss
Netid State      Recv-Q Send-Q                                  Local Address:Port                                                   Peer Address:Port                
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported
Cannot open netlink socket: Protocol not supported

师傅您好,我最近也在用树莓派平台,运行docker,跑arm、x86等其他跨平台的系统。 您的这个ps无法运行,其他命令能运行吗?如果都无法运行,估计是依赖库的问题吧。[email protected],再交流一下?

zhjygit avatar Apr 07 '23 01:04 zhjygit