buildah
buildah copied to clipboard
containers
3.0.0, 2.1.1: rootless Podman cant run "expect" (The system has no more ptys.) [regression]
Since few releases ago (going several weeks back) I can't (re-)build a particular container image any more due to expect binary failing (in spawn command) as follows in rootless mode:
The system has no more ptys. Ask your system administrator to create more.
(Formerly this container image was building fine by the older release of Podman).
expect(1) man page mentions the following:
Internally, spawn uses a pty, initialized the same way as the user's tty.
# ls -l /dev/pts
total 0
crw--w---- 1 root tty 136, 0 Feb 7 14:18 0
crw-rw-rw- 1 root root 5, 2 Feb 7 14:18 ptmx
I'm not sure whether this have something to do with kernel, runc, crun or other system components.
I've tried with runc and crun but reproduced the problem with both of them.
Podman 2.0.6 on Debian "testing"/"unstable" had no such problem.
Here is the current output of podman info:
host:
arch: amd64
buildahVersion: 1.19.3
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: 'conmon: /usr/bin/conmon'
path: /usr/bin/conmon
version: 'conmon version 2.0.25, commit: unknown'
cpus: 4
distribution:
distribution: debian
version: unknown
eventLogger: journald
hostname: deblab
idMappings:
gidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 427680
size: 65536
uidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 427680
size: 65536
kernel: 5.10.0-2-amd64
linkmode: dynamic
memFree: 1301323776
memTotal: 67363360768
ociRuntime:
name: crun
package: 'crun: /usr/bin/crun'
path: /usr/bin/crun
version: |-
crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
os: linux
remoteSocket:
exists: true
path: /run/user/1000/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
selinuxEnabled: false
slirp4netns:
executable: /usr/bin/slirp4netns
package: 'slirp4netns: /usr/bin/slirp4netns'
version: |-
slirp4netns version 1.0.1
commit: 6a7b16babc95b6a3056b33fb45b74a6f62262dd4
libslirp: 4.4.0
swapFree: 15508434944
swapTotal: 17495486464
uptime: 14h 13m 26.36s (Approximately 0.58 days)
registries:
search:
- docker.io
- registry.fedoraproject.org
store:
configFile: ~/.config/containers/storage.conf
containerStore:
number: 0
paused: 0
running: 0
stopped: 0
graphDriverName: overlay
graphOptions:
overlay.mount_program:
Executable: /usr/bin/fuse-overlayfs
Package: 'fuse-overlayfs: /usr/bin/fuse-overlayfs'
Version: |-
fusermount3 version: 3.10.1
fuse-overlayfs: version 1.4
FUSE library version 3.10.1
using FUSE kernel interface version 7.31
graphRoot: ~/.local/share/containers/storage
graphStatus:
Backing Filesystem: btrfs
Native Overlay Diff: "false"
Supports d_type: "true"
Using metacopy: "false"
imageStore:
number: 15
runRoot: /run/user/1000
volumePath: ~/.local/share/containers/storage/volumes
version:
APIVersion: 3.0.0
Built: 0
BuiltTime: Thu Jan 1 10:00:00 1970
GitCommit: ""
GoVersion: go1.15.7
OsArch: linux/amd64
Version: 3.0.0-rc2
CC: @siretart.
Can you provide additional details on how you're launching Podman - an exact command line to reproduce would be greatly helpful.
Something like podman build --no-cache --rm --force-rm -t centos7-app .
Build script invokes expect script that fails... It is a straightforward script:
#!/usr/bin/expect
set timeout 99
stty columns 80 rows 25
spawn bash /var/tmp/appsetup-linux.sh
sleep .5
expect "Please enter the fully qualified name, including domain information, of this host machine*" {
send "localhost\n"
}
interact
It is invoked as runuser -u appuser /var/tmp/appsetup-linux.expect during container build.
@onlyjob Could you try this against buildah in both rootful and rootless mode. Also have you tried this against the podman 3.0 rc?
In podman 3.0.0~rc2 (mentioned in title) it appears to work under root. I'll check if buildah is affected and report...
If you run that command in your user session, non root, do you run out of the ptys as well?
If you run that command in your user session, non root, do you run out of the ptys as well?
Apparently not... expect don't complain under podman run -it (rootless)...
Basically -i keeps stdin open. But it might do something with the tty.
You could so try this with crun and runc, to see if they react any differently.
A friendly reminder that this issue had no activity for 30 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
A friendly reminder that this issue had no activity for 30 days.
You could so try this with crun and runc, to see if they react any differently.
I did not have a chance to try that. The problem manifested with crun. Did you have a chance to reproduce?
Have not been able to reproduce, please try it against current code, and /or generate a reproducer we can test against.
A friendly reminder that this issue had no activity for 30 days.
A friendly reminder that this issue had no activity for 30 days.
Since we have had no feedback in a month, I am going to close. Reopen if you have the feedback.
I can only comment but not reopen... I'm sorry that I could not provide a perfect reproducer (no time!) but this bug report is complete with everything that is needed for replicating the issue.
Minimal reproducer:
FROM debian:unstable
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
expect \
&& rm -rf /var/lib/apt/lists/*
RUN expect -c 'spawn echo 1'
Build with:
podman build --network host -f test9_dockerfile someemptydir
Expected result:
Does not print:
STEP 3/3: RUN expect -c 'spawn echo 1'
spawn echo 1
The system has no more ptys. Ask your system administrator to create more.
while executing
"spawn echo 1"
relevant strace excerpt from a similiar but different setup:
write(1, "spawn", 5spawn) = 5
write(1, " ", 1 ) = 1
write(1, "1", 11) = 1
write(1, "\r\r\n", 3
) = 3
openat(AT_FDCWD, "/dev/ptmx", O_RDWR) = 4
ioctl(4, TIOCGPTN, 0x7ffcfee9a1f4) = -1 EACCES (Permission denied)
close(4) = 0
close(-1) = -1 EBADF (Bad file descriptor)
close(-1) = -1 EBADF (Bad file descriptor)
openat(AT_FDCWD, "/", O_RDONLY) = 4
close(4) = 0
write(2, "The system has no more ptys. As"..., 105The system has no more ptys. Ask your system administrator to create more.
while executing
"spawn 1") = 105
This might very well be a bug in expect, https://sources.debian.org/src/expect/5.45.4-2/exp_command.c/?hl=873#L873 is where the error message is printed.
It seems that the code responsible is here https://sources.debian.org/src/expect/5.45.4-2/pty_termios.c/#L390
And a (horrible) C code reproducer:
#include <assert.h>
#include <stdio.h>
#include <fcntl.h>
#define __USE_XOPEN_EXTENDED 1
#include <stdlib.h>
int main(int argc, char *argv[]) {
int master = open("/dev/ptmx", O_RDWR);
if (master == -1) {
return EXIT_FAILURE;
}
char * v = ptsname(master);
printf("ptsname => '%s'\n", v);
return EXIT_SUCCESS;
}
in podman:
openat(AT_FDCWD, "/dev/ptmx", O_RDWR) = 3
ioctl(3, TIOCGPTN, 0x7ffc88b3bdfc) = -1 EACCES (Permission denied)
outside:
openat(AT_FDCWD, "/dev/ptmx", O_RDWR) = 3
ioctl(3, TIOCGPTN, [44]) = 0
Please disregard the above, I managed to block the TIOCGPTN ioctl and didn't double check.
For some reason I was expecting SELinux to block with EPERM not EACCES, and I didn't double check first.
Sorry for the noise.
A friendly reminder that this issue had no activity for 30 days.
That's a shame. I've hit this issue trying to build an Oracle database container using Podman.
