htmap
htmap copied to clipboard
htcondor
High-Throughput Computing in Python, powered by HTCondor
**What's your issue?** I have launched several (supposedly) short jobs. On EC2 with a modern NVIDIA GPU, they take around 40 minutes. I have launched these jobs on HTCondor, and...
i.e., `Map.wall_time`, expose in the CLI status table, etc.
It would be convenient to get a long-form job ad (`condor_q -long`) for a particular map component via HTMap's interface. Should be possible from both the Python API and the...
In #198 , @keith6014 noted that > I also did this > > ```python > htmap.settings['MAP_OPTIONS.executable']=sys.executable > ``` > > The job still gets held. Same held reason. But I...
While working on https://github.com/msu-coinlab/pymoo/pull/46 , it occurred to me that it would be convenient to be able to mark a map as "super-transient": as soon as it's iterated, it gets...
**Is your feature request related to a problem? Please describe.** I have a job that requires a GPU. It requires a GPU and specifies a Docker image. I need to...
In particular, `cloudpickle`. If `cloudpickle` is missing execute-side, we error out before managing to do any error handling and don't provide a useful error message to the user. The component...
Right now, we pre-process all of the map data in memory while writing the map out to disk. The primary offender is https://github.com/htcondor/htmap/blob/5be40dd2577b73cd657ff955479fdb33d77f176e/htmap/mapping.py#L433, but the ease of using the pre-processed...
It should be possible to detect some obvious situations where, without user config, we should set "assume" delivery. @tannenba suggests a simple check: if the filesystem and uid domains are...
Creating a map with a very large number of components (>10k complex input objects is when I've seen it), can take several minutes. We should give some kind of feedback...