law
law copied to clipboard
riga
Build large-scale task workflows: luigi + job submission + remote targets + environment sandboxing using Docker/Singularity
The Pull Request (PR) introduces the possibility of handling targets in HDFS (Hadoop Distributed File System). The structure of the new classes mimics the GFALFileInterface and WLCGFileSystem ones. New classes:...
[](https://github.com/riga/law/actions/workflows/lint_and_test.yml) - [x] Update project packaging - [x] Drop Python 2 support - [ ] Polish code (remove deprecated features, type hints, language features) - [x] law - [x]...
When dropping Python 2 support for the next release, the custom config interpolation should be removed (deprecated at first) and replaced by the [built-in extended interpolation](https://docs.python.org/3/library/configparser.html#configparser.ExtendedInterpolation).
Write doc strings and the structure for the doc pages. **Pages** - [ ] Tasks - [ ] Targets - [ ] Workflows - [ ] Sandboxing - [ ]...
Way more logs are required to get a sense of what is happening in the background if required. Therefore, most of the logs might have DEBUG level, but certainly some...
In some environments, gfal2 and its python bindings might require manual hacks to run. It might be useful to have a shim that transfers the gfal2 API calls to a...
A jupyter notebook could be helpful to document the most important aspects of law and to give some deeper, practical insights into tasks, targets, filesystems, etc. This could be launched...
- [x] dropbox_targets - [ ] wlcg_targets - [ ] docker_sandboxes - [ ] wlcg_at_cern
In some places it could make sense to use custom exceptions, e.g. to allow users to catch some types and continue.