pysqa
pysqa copied to clipboard
pyiron
[feature] Extend the executor support
Add support for resource definition during submission time and function dependence in analogy to pympipool:
- https://pympipool.readthedocs.io/en/latest/examples.html#resource-assignment
- https://pympipool.readthedocs.io/en/latest/examples.html#coupled-functions
Especially in combination with the remote submission using SSH it is important to synchronise the local and the remote cache. To make this cache visible to the user it would be great to create a sub-folder in the current working directory locally. The important part is to also have the ability to link to the output of a previous calculation.
Example calculation for demonstration would be the calculation of a band structure with VASP. This requires two calculation with the WAVECAR being copied from the first to the second calculation. I think the separation of temporary cached data during the execution of the workflow and the storage of the results at the end of the workflow are two very separate tasks.
An example based on executorlib and pysqa is available in https://github.com/pyiron-dev/remote-executor