smartdispatch
smartdispatch copied to clipboard
SMART-Lab
An easy to use job launcher for supercomputers with PBS compatible job manager.
It would be nice to have some guidelines on how to extend smartdispatch to use new clusters.
It would be really nice to handle different type of folded arguments. Here a list of what I think could be useful. Feel free to comment or suggest other types....
I made a few updates to make smart-dispatch compatible with Python 3, namely replacing all print statement calls by the print function, converting a bytestring to a string and using...
Fixes #157. Currently the user's entire environment is exported. This is bad practice because it makes jobs hard to reproduce and debug, and in some cases it can even cause...
Currently smart-dispatch exports the current environment to the job using `-V`. This is bad practice, see e.g. McGill's HPC guidelines: > We no longer recommend adding corresponding "module load ..."...
- [x] Also make sure that the jobs are appended properly to jobs_id.txt when we resume. - [x] Allow using both path and Batch UID to resume jobs. - [x]...
I really like using smart_dispatch and I have a few ideas how it could be improved: - The location for creating logs should configurable, now `SMART_DISPATCH_LOGS` seems to be hardcoded....
If you ask for more resources than what is available on the requires queue we just crash. We should handle that gracefully and throw a proper error message. See #124...
- [ ] Add queues - [ ] Make sure it uses msub - [ ] Make sure it returns the right ID. See #137