HEMTT icon indicating copy to clipboard operation
HEMTT copied to clipboard

[WIP] Dependency graph

Open bovine3dom opened this issue 5 years ago • 5 comments

For some mad reason, I decided I wanted to do this properly. It therefore might take me a little while. I'm opening this PR mostly to let people keep an eye on it and tell me if I'm going very off-piste.

I've got up to the point where I have a big vector of all the processed config.cpps.

I could just do a text search for CfgPatches and requiredAddons[], but since armake2 has to make a definition map anyway, it probably makes sense to try to get at that: https://github.com/KoffeinFlummi/armake2/blob/40fabd915514ffda372ec012b35ed4190d0e0515/src/preprocess.rs#L438

I'll probably do it the text search way first, as getting at that map without copy-pasting a large amount of code will probably require a PR to armake2.

Further down the to-do list is to ask armake2 to stop once it has got up to CfgPatches.requiredAddons[] in each file. (If someone defines requiredAddons more than once in a single config, they don't deserve my graphs anyway). I suspect that would speed stuff up a lot.

Closes #54 once finished.

bovine3dom avatar Mar 12 '19 22:03 bovine3dom

Mostly working now with the following caveats:

  • [x] Doesn't work with multiline requiredAddons
  • [x] need to trim the requiredAddons strings as apparently they include spaces sometimes
  • [ ] need to store and compare the node indices using lowercase as requiredaddons is apparently case insensitive

I'd quite like to do a similar thing for showing the graph of classnames mentioned in each addon, as I'm pretty sure we have quite a few secret dependencies.

bovine3dom avatar Mar 19 '19 23:03 bovine3dom

You could try using this to remove your regex, then use serde_json to turn it into a rust structure. I was probably going to include that at some point anyway.

BrettMayson avatar Mar 20 '19 23:03 BrettMayson

That would improve the current PR, but I'd like to eventually dig into armake to allow us to have early stopping, which would lead to invalid classes which I suspect your jsonifier wouldn't like. I suppose I should have a go at doing that first.

bovine3dom avatar Mar 28 '19 21:03 bovine3dom

I mentioned the idea of hemtt bisect before, like git bisect but bisecting and descending trees until it found the folder that caused an error. We could similarly use it to create debug releases of a subset of addons and all of their dependencies. hemtt build --recursive [addons] seems like a sensible interface to me.

bovine3dom avatar Mar 29 '19 14:03 bovine3dom

Added early stopping for a respectable speedup:

On a large project (~130 addons):

bash-5.0$ time ~/projects/HEMTT/target/debug/hemtt dependencies > old_version.dot

real    0m36.541s
user    1m40.267s
sys     0m0.397s
bash-5.0$ time ~/projects/HEMTT/target/debug/hemtt dependencies > early_stopping.dot

real    0m10.555s
user    0m35.824s
sys     0m0.143s

On a small project (with four addons):

bash-5.0$ time ~/projects/HEMTT/target/debug/hemtt dependencies > old_version.dot

real    0m0.487s
user    0m1.154s
sys     0m0.010s
bash-5.0$ time ~/projects/HEMTT/target/debug/hemtt dependencies > early_stopping.dot

real    0m0.409s
user    0m0.996s
sys     0m0.007s

Next stop - coaxing more information out of armake2's parser.

Edit: speedup is even more obvious on a release build on a large modpack -

bash-5.0$ time ~/projects/HEMTT/target/release/hemtt dependencies > old_version.dot

real    0m5.567s
user    0m14.692s
sys     0m0.357s
bash-5.0$ time ~/projects/HEMTT/target/release/hemtt dependencies > early_stopping.dot

real    0m0.372s
user    0m1.127s
sys     0m0.110s

I was considering caching the graph but it's basically quick enough for interactive usage.

bovine3dom avatar Apr 03 '19 21:04 bovine3dom