Optionally make exec session terminate with parent

[ghost]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

Add a flag to podman exec to make exec session terminate with parent, similar to bubblewrap's bwrap --die-with-parent.

Immutable distributions make use of toolbox / distrobox to provide a mutable environment. A common use is to run commands directly within container (toolbox run [COMMAND] / distrobox enter -- [COMMAND]), since they use exec session, they have the same limitation of not terminating child-proceess when terminal emulator is closed.

Steps to reproduce the issue:

1. Open System Monitor / Task Manager equivilent in your desktop environment, search for sleep

2. Run the following command in your terminal emulator (either one will work):

podman:

podman run --rm -it \
    --name debian \
    --entrypoint /bin/sh \
    docker.io/library/debian:11

# In a new terminal emulator window
podman exec debian sleep 30

toolbox:

toolbox create
toolbox run sleep 30

distrobox:

distrobox create
distrobox enter -- sleep 30

3. Then try to close terminal emulator, it'll prompt something like this:

4. Insist closing it, then look at System Monitor

Describe the results you received:

sleep 30 still runs within container.

Describe the results you expected:

Nah, this is expected, hence this feature request.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Client:       Podman Engine
Version:      4.2.1
API Version:  4.2.1
Go Version:   go1.18.5
Built:        Thu Sep  8 03:58:19 2022
OS/Arch:      linux/amd64

Output of podman info:

Click me

host:
  arch: amd64
  buildahVersion: 1.27.0
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.4-3.fc36.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.4, commit: '
  cpuUtilization:
    idlePercent: 60.53
    systemPercent: 23.27
    userPercent: 16.2
  cpus: 4
  distribution:
    distribution: fedora
    variant: silverblue
    version: "36"
  eventLogger: journald
  hostname: fedora
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 6.0.5-200.fc36.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 224858112
  memTotal: 16705081344
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.6-2.fc36.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.6
      commit: 18cf2efbb8feb2b2f20e316520e0fd0b6c41ef4d
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-0.2.beta.0.fc36.x86_64
    version: |-
      slirp4netns version 1.2.0-beta.0
      commit: 477db14a24ff1a3de3a705e51ca2c4c1fe3dda64
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 28844679168
  swapTotal: 34359734272
  uptime: 42h 55m 22.00s (Approximately 1.75 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /var/home/user/.config/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 1
    stopped: 1
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/user/.local/share/containers/storage
  graphRootAllocated: 510389125120
  graphRootUsed: 201430126592
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 104
  runRoot: /run/user/1000/containers
  volumePath: /var/home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 4.2.1
  Built: 1662580699
  BuiltTime: Thu Sep  8 03:58:19 2022
  GitCommit: ""
  GoVersion: go1.18.5
  Os: linux
  OsArch: linux/amd64
  Version: 4.2.1

Package info (e.g. output of rpm -q podman or apt list podman or brew info podman):

podman-4.2.1-2.fc36.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

• Latest: No
• Troubleshooting: Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Fedora Silverblue 36

[mheon]

It's more accurate to say that we can't make them not terminate when the container exits. The kernel enforces the rule that any PID namespace will kill every process in the namespace if PID 1 in the namespace dies; Podman will take down PID 1, guaranteeing that the kernel will unwind the rest of the namespace. For containers without a PID namespace, it's a bit trickier, but we do have an accurate list of processes in the container, which we then individually kill as part of stopping the container. In short, I strongly doubt your Podman reproducer actually does what you think it does; the kernel simply won't allow that to happen.

[ghost]

Sorry if i didn't explain it deeper. I'm not talking about the process being lingering when container is stopped, as this had never been the case, just as you said.

With toolbox / distrobox executing commands inside of container, the container is NOT stopped after the command is finished.

And the parent I'm talking isn't PID 1 of the container, but the podman exec process in terminal emulator. I guess a better term should be used here, since structurally podman exec process isn't a direct parent of container process.

This is a screenshot which represent the issue better:

If I try to close the terminal emulator, it'll prompt the following:

If I press "Close Terminal", sh, toolbox and podman (which runs exec command) will be terminated because they're child process of the vte session.

However, notice the conmon and its child process sleep aren't part of gnome-terminal-server. When sh is terminated, podman (exec command) will be terminated but the corresponding conmon process will be kept intact. As a result, sleep 30 isn't terminated properly.

And sleep 30 is only used for demonstration. In reality one could run something resource intensive, and then close the terminal emulator not knowing they're lingering in the background.

This is probably only an issue for pet container usecase. toolbox / distrobox tends to start a trap program inside container to keep it running. Anything interactive is executed by podman exec, hence this issue.

The feature request, to be precise, is to add an optional flag that make the conmon process terminates when the corresponding podman process is dead.

[Luap99]

@mheon I think the request is basically to not double fork conmon and not let it create a new process group to keep it attached to the podman parent process.

[ghost]

Yes. This would work as well.

[mheon]

@Luap99 Don't know if that works. Conmon dying is only going to take out the first PID the exec session started; anything else it did, probably just reparents on top of PID 1 in the container. So we can definitely kill a single-process exec session, but a podman exec -ti $ctr bash like Toolbox does, we only get bash, not anything bash was doing (unless the shell automatically kills its children on exit, not something we can guarantee for every program).

We don't really have a robust way of tracking what processes were spawned from an exec session right now. We'd basically have to walk the process tree in the container, which seems potentially racy. On CGv2, a child cgroup might be a solution? Just need to make sure it doesn't interfere with the container itself being stopped...

[rhatdan]

I believe it walks the cgroup and kills all of the pids within the cgroup, or at least I remember this is what we wrote many years ago.

[debarshiray]

I wonder if Toolbx could detect this scenario and explicitly terminate the process that it had launched inside the container.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

This seems like more of an issue for toolbx rather then podman.

[debarshiray]

This seems like more of an issue for toolbx rather then podman.

Umm... it's not really clear to me what Toolbx could do here. Is there a recommended way to get to the process ID of the conmon process?

[debarshiray]

@Luap99 Don't know if that works. Conmon dying is only going to take out the first PID the exec session started;

I think it's good enough if conmon died and took out the first PID that the exec session started, because ...

anything else it did, probably just reparents on top of PID 1 in the container. So we can definitely kill a single-process exec session, but a podman exec -ti $ctr bash like Toolbox does, we only get bash, not anything bash was doing (unless the shell automatically kills its children on exit, not something we can guarantee for every program).

... if this was a shell directly running on the host without involving any containers, the expectation is that closing the terminal emulator takes out the shell and anything that's willing to die with it. If someone started a process in the background (say, sleep +Inf &), then it's OK if it keeps running in the background.

[debarshiray]

@giuseppe ameliorated one problematic outcome of this - the processes inside the exec sessions blocking shutdown. See https://github.com/containers/podman/pull/17025

However, it's still worth trying to ensure that the processes inside the exec session goes away as soon as the terminal emulator is closed, just as it happens when one is working directly on the host.

I have to say that I am a bit puzzled that the processes are outliving their controlling terminal. I know there's an inner nested terminal device for the container, but isn't it supposed to go away with the outer terminal?

[debarshiray]

This seems like more of an issue for toolbx rather then podman.

Umm... it's not really clear to me what Toolbx could do here. Is there a recommended way to get to the process ID of the conmon process?

@mheon @rhatdan @Luap99 @giuseppe Could one of you please help answer this question?

We are brainstorming various options at https://github.com/containers/toolbox/pull/1207 but it's not clear if it's possible for the podman exec caller to get the process ID of conmon(8) or the process inside the container.

Also, it's not clear to me why podman exec --interactive --tty should not terminate the foreground container process with it. Especially when podman exec -it is getting terminated by a SIGHUP from its controlling terminal.

[giuseppe]

I wonder if it will be easier for you to just use the OCI runtime to do the exec.

e.g. if you do crun exec you circumvent podman and conmon, I am fine to add something like --die-with-parent to crun in a similar way to what bwrap does.

Can you please play with it and see if "crun exec" does all you need?

Adding it to Podman/conmon will be much more complicated, we will need to change the way conmon works to not perform a double fork.

[Luap99]

That said podman run is forwarding all signals (well the ones that can be caught) into the container so maybe should podman exec do that to.

ref https://github.com/containers/toolbox/issues/1400