nomad icon indicating copy to clipboard operation
nomad copied to clipboard

exec: allow running commands from host volume

Open tgross opened this issue 3 years ago • 2 comments

Fixes https://github.com/hashicorp/nomad/issues/11781

The exec driver and other drivers derived from the shared executor check the path of the command before handing off to libcontainer to ensure that the command doesn't escape the sandbox. But we don't check any host volume mounts, which should be safe to use as a source for executables if we're letting the user mount them to the container in the first place.

Check the mount config to verify the executable lives in the mount's host path, but then return an absolute path for the mount's task path so that we can hand that off to libcontainer to run.


DRAFT: the implementation here is incomplete and a little gross in terms of passing around the "from mount" flag. We still need to do the relative path safety check inside the mount. But right now this is a proof-of-concept that @mikenomitch nerd-sniped me into doing, and I can polish this into a real fix for this long-standing papercut.

tgross avatar Oct 07 '22 19:10 tgross

Would close https://github.com/hashicorp/nomad/issues/11781

mikenomitch avatar Oct 07 '22 20:10 mikenomitch

The exec driver and other drivers derived from the shared executor check the path of the command before handing off to libcontainer to ensure that the command doesn't escape the sandbox

Is the only reason for this to "speed up" errors due to the executable not being reachable to pre-chroot instead of post? Seems like just letting the command fail to execute in the chroot would fix this surprising behavior and simplify code at the cost of "slower" task failures?

If we used an overlayfs we could remove that slowdown for the failure case as well as the happy path.

schmichael avatar Oct 07 '22 21:10 schmichael

Is the only reason for this to "speed up" errors due to the executable not being reachable to pre-chroot instead of post?

It turns out we also set the executable bit on the command binary for the user. For the exec driver (or any driver with FSIsolationChroot instead of FSIsolationImage) the task dir includes copies of the client.chroot_env. So it's safe to set the executable bit... so long as we make sure that we're doing so to one of those files copied into the task dir sandbox. Or now, in the mounted host volume.

tgross avatar Nov 09 '22 21:11 tgross

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

github-actions[bot] avatar Mar 12 '23 02:03 github-actions[bot]