Use shell executable versions of compression and hash algs instead of rust versions
Is your feature request related to a problem? Please describe.
Our current system of detecting hashing and compression algorithms is inflexible. We need to keep adding new algs on (like #23, #119, #130), the hard linkage causes development headaches (#144, #156), the code for them is really cursed because it's a pair of ginormous macros, each library has a slightly, yet annoyingly different interface, they're slower than they should be due to the fact that I need to deal with the parallelism manually, there's potential security risks if I don't keep these versions constantly updated...
Describe the solution you'd like
Drop all of our compression and hash algorithm dependencies. We will call their respective shell executables (gunzip, xzcat, sha256sum...) to use those algorithms. We'll still auto-detect if the file is a xz or a bz, or if we have sha256 or md5 in the directory, but the user should be able to provide their own commands when we don't know about those.
We'll want to print absolute, de-unicoded paths to the executables we've detected so that they don't get homograph'd (like with /usr/biո/xzcat (that's an armenian ո by the way)) or wrong-PATH'd (oops we found ~/bin/blegh but you wanted /usr/bin/blegh), and ask for the user's approval.
Then, all of our distros are going to need to add our compression algorithm executables as either required or recommended dependencies.
Why is the code like this to begin with?
- Premature optimization. I thought it would be more efficient to inline this inside the file. However, if you're just writing to external disks, the bottleneck will almost always be the disks, and not whether or not your compression/hash implementation is inlined, so that's kinda silly.
-
Portability. I originally envisioned Caligula being run on embedded or resource-constrained environments where, for some reason, you can't even copy an
xzbinary or something on. Also I envisioned maybe you don't have tools likexzinstalled. However, the primary use case is having desktops write to external drives, and if you don't havexzinstalled... you can just install it? Besides, if you are trying to do something on an environment you can't copyxzto, you probably can't copy caligula on it either, so just useddat that point.
I just want to mention that this may cause problems with #2 Windows (or any other non *nix OS) may not have those compression tools / may not have the same flags/args.
I'm not opposed per se (especially since windows support is a long standing problem) but I just wanted to point that problem out before much work is put into this.