Get a shell in the environment/sandbox of an existing service

[Lucus16]

Component

systemd-run

Is your feature request related to a problem? Please describe

I'm often trying to debug a service where I don't understand what the environment of that service looks like. In that case I'd want to be able to spawn a shell with exactly the environment (variables, user, group, mounts, other sandbox features) specified by a service so I can try some commands in it and see the results directly. Especially when the service runs in a sandbox, I can't easily peek inside and I can't easily become the right dynamic user. Being able to spawn a shell in the same sandbox would help me a lot to find issues in the sandbox without needing to take the service offline.

I often have manual maintenance I want to do in the same environment as that of a service. I want to be sure I write files as the right user, don't accidentally write to paths I shouldn't, etc... That would all be trivial if I could start a shell in the same environment as the target service.

Describe the solution you'd like

Ideally, there's a dedicated systemd-shell foo.service command or a flag I can pass to systemd-run like systemd-run --shell --in-environment-of=foo.service. That would spawn a shell (either the one from the user corresponding to the service if it has a usable one, or a global one, or one explicitly passed as argument) which matches the runtime environment of the service as closely as possible. It would also use the working directory specified in the service. If the service was already running, it would use the same (possibly dynamic) user as the service and generally be in the same sandbox such that it can see everything the service sees. If the service is started after such a shell exists, the service would join and keep the sandbox of the shell.

Describe alternatives you've considered

I can run things as root and then manually patch up the owner and group files I create, but the odds of making mistakes are large. I can try to get closer to the environment of service unit by manually applying each of the many settings, e.g. starting with the user and group and all the environment variables, but it would still be quite different because systemd units can also set bind mounts and have various other sandbox features. It would also be a lot of work to reproduce manually. I don't know much about slices but it might be possible to give each service its own slice, define all sandboxing features in that slice and then use systemd-run --shell --slice=foo.slice but it would require me to rework all my services and to split their configuration between the slice and the service unit.

The systemd version you checked that didn't have the feature you are asking for

253.3

[bluca]

Have you tried nsenter?

[Lucus16]

That does seem to help a lot. But it can't help me debug a systemd service that fails immediately and I'm not sure if it can replicate all parts of the environment that can be described in a service file.

[bluca]

for that use systemd-run

[Lucus16]

I found the systemd-run --shell option but I don't see any option in the manual to base the environment on an existing service. For clarity, I do not mean just the environment variables, but also user, group, mounts and all the various sandbox features.

[bluca]

Just pass them with --property

[Lucus16]

While that is possible, it can easily be 20 things I need to copy paste without making a mistake. The feature I'm requesting then is a single flag that takes a service name or filepath as argument in order to do that automatically. Call it --properties-from for example.

[dechamps]

I second this request - this would be quite useful.

Have you tried nsenter?

nsenter brings one pretty close to emulating the sandbox the service processes are running in yes, but there are still gaps. For example, nsenter will not reproduce system call restrictions, capabilities, resource limits, to name just a few.

[vcaputo]

A service equivalent of machinectl shell would be useful.

[stephank]

I currently use a convenience script like:

#!/bin/bash
pid="$(systemctl show --property MainPID --value myapp.service)"
exec sudo nsenter -t "$pid" -a --wd=/opt/myapp -e -S follow -G follow bash

Two other nuances I found:

• It inherits TERM=dumb, but maybe specific (hardcoded?) environment variables should be inherited from the parent process, instead of the target process.
• It inherits environment variables from the target process that may not be set in the unit. This caused some unexpected behavior for me.

[nogweii]

Is this a duplicate of #9907? Being able to run a shell as if it were within the configued sandbox of a systemd service is pretty close to actually running a shell within the same cgroups of a running service.

And if you really did need to be in the exact same cgroups, nsenter is there.

[Lucus16]

Passing the service file of the target service would be an okay solution for this. As mentioned before, nsenter won't help in the case where the service isn't running.