htmap
htmap copied to clipboard
htcondor
High-Throughput Computing in Python, powered by HTCondor
The `Settings` object is way too complicated for what it actually needs to do. New version should - Support only one level of nesting (keys are either values, or dictionaries...
Presumably I'm missing some state transitions that exist in the event logs in this HTMap dir: https://www.dropbox.com/s/m2q748xuz944p1r/htmap-missed-events.tar.gz?dl=0 Vacating all of the maps and letting them restart brought them back in...
So that you can slice over the components of a map as if it was a sequence, which we claim it is anyway... Possibly also special support for `map[ComponentStatus.Completed]` to...
HTMap's singularity support currently relies on the execute node being able to do overlays, because we bind the scratch directory to a target that (probably) doesn't exist. It would be...
The transplant dir is set during startup, unlike the maps dir, which is constructed at runtime. Should probably just do it the same way as the map dir. Not so...
It should be possible to transfer a map to another user. Some connection to #47 , which will allow an imported map to be re-run by replacing certain submit variables...
In particular, paths to the HTMap library itself and to files in the map directory should be interpolated by another submit variable, where possible.
This will fail occasionally with something like `PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\joshk\\.htmap\\maps\\foo\\cluster_logs\\253.log'`: ```python import htmap htmap.force_clean() @htmap.htmap...
There's significant overhead in running very short jobs, so there should be a way to batch up multiple inputs into a single job. The user just sets the number of...