brainstorm ways to augment path-resolution, improve runtime isolation

[abathur]

In #4 @grahamc made two more suggestions that, as I currently understand them, both boil down to: more resolution! more isolation! I'd like to try exhausting the solution space a bit, and spend some time hammering on the solutions to see which ones are promising enough to seriously consider. I'll try to keep the first post up-to-date as a summary...

I'm not exactly sure what is and isn't (behaviorally?) in-scope for resholved, yet; some of these will almost certainly be out-of-scope. Don't see these as either-or propositions; a good approach might combine a few.

More aggressive resolution modes

1. Mode that treats all "words" as potential commands, tries to resolve every single one, and requires the user to triage any that resolve.

Inject shell at the head of the script to unset PATH

(or, more likely, set it to the empty string).

1. The ideologue's approach is probably something like declare -xr PATH= (export & readonly).

   This is a very Nix approach to the problem, but it also entails a commitment to patching anything it breaks. I imagine a great many edge cases around scripts that are sourced, or get cat-ed together.

2. A "gentle" approach would just empty PATH.

   Some scripts have sanity-checking routines of various quality that may discover the empty/weird path and helpfully re-set it to something more normal.

Modify the script to instrument it for data gathering

1. Trace the execution?

   • Not sure how much we can meaningfully do at the bash level. There's a hook for fielding unresolved commands, command_not_found_handle, but resholved hopefully gets most of the low-hanging fruit here and I assume many edge-cases will be external command behavior (which this wouldn't catch), so I'm not sure how much it would help in practice. Probably more tractable at the system level, but not sure it's very cross-platform? IIRC trace is very locked-down on macOS?

   • As @catern mentions in #13, it may be possible to point PATH at a fuse filesystem and run the script to get some extra leverage here. This sounds promising, since it would be cross-compatible. I'm not sure if this would be in resholved's wheelhouse, or be an additional tool?

   • I had a separate thought late last night about an isolated runtime environment that uses some sort of fuzzer approach to try and exercise as many logic paths as feasible: rewrite the script to replace all of the identifiable tokens with aliases or functions that randomly:

     • return true or false
     • echo from a deliberately chosen list of formats or run the true underlying executable

     And then, of course, run the script many times. Maybe tune the number of repetitions by trying it out on some places where we've affirmatively found issues in the past. Will still be a little helpless with conditional logic on exact variable/argument contents.

   At some point, we're inevitably going to start bumping up against things that are unknowable/unprovable due to how malleable shell is in the first place or won't fall out of the tree unless we say the magic words in the right order. My own heuristic for how resholved should approach cases it can't directly solve for is to focus on finding a reliable heuristic for spotting the problem(s), raise an error, and make the user tell resholved what to do about it.

[abathur]

@grahamc I've added WIP support for inserting code at the beginning (or end) of a script in #12. You can try it out from commit https://github.com/abathur/resholved/pull/12/commits/3bd1b01546bc34ff5258adeb3cdca859697bb1d9 by passing a flag like --insert-before-script "declare -xr PATH="

I've also been trying to think through which distinct cases are probably under the umbrella here:

1. resholved didn't undertstand the syntax well enough to realize something is a command a. it was supplied in inputs/RESHOLVE_PATH (but obviously not resolved) b. it wasn't supplied in inputs (the user had no chance to notice because resholved never objected to its absence).
2. a script or executable invoked or sourced by a resholved script has its own unidentified/unpatched dependency on some other executable that it expects to find on PATH or at an absolute path (I already stumbled on one of these myself in NixOS/nixpkgs#89063)

If you already have or run into some fresh real-world examples (of both the shell and Nix inputs), they may help clarify which options are most likely to help.