make executable.storage contain exec text itself?

[appassionate]

trafficstars

Hi! pyiron handle the total submit script as run_queue.sh and "bin/executable.sh" in resource path. Is it possible to read the executable.sh content in resource path and save it to the executable.storage? Since the executable.sh is just an very short text, if job submitted in remote resource by pysqa, another executable in the remote should be pre-determined in remote resource path, which seems to be a little inconvient.

thanks.

[jan-janssen]

@appassionate the idea of having the remote executables is that a research group could share them between all users, so that they only need to be defined once. Still we never got around to implement an automatic look up of executables on a remote compute clusters. In the same way we could handle the remote look up of queuing systems. Finally, one argument to keep the executables located on the cluster is to be able to debug pyiron from the command line when necessary.

[appassionate]

@appassionate the idea of having the remote executables is that a research group could share them between all users, so that they only need to be defined once. Still we never got around to implement an automatic look up of executables on a remote compute clusters. In the same way we could handle the remote look up of queuing systems. Finally, one argument to keep the executables located on the cluster is to be able to debug pyiron from the command line when necessary.

Thanks for your detailed reply!

My point is that job.executable actually can be translated to a text after processed by pyiron job method. The current implementation is that executable.storage still contain some "stateful variable" (in my opinion) such as storage.version, storage.name which is related to a special resource path in some hpc-cluster. It does work very well, but the most direct way to set executable might be just writing out a exec.sh file (from resource template) by job.executable.storage into job working_directory.

[pmrv]

Just to clarify (I'll likely won't get to working on it soon):

In your proposed model it would go roughly like this:

1. Job is submitted/run
2. pyiron looks up executable script from resources and saves them in HDF of job
3. pyiron writes queue submission script which calls python -m pyiron ...
4. that code then writes the saved script from HDF into the working directory and runs it

[appassionate]

Just to clarify (I'll likely won't get to working on it soon):

In your proposed model it would go roughly like this:

1. Job is submitted/run
2. pyiron looks up executable script from resources and saves them in HDF of job
3. pyiron writes queue submission script which calls python -m pyiron ...
4. that code then writes the saved script from HDF into the working directory and runs it

Yeah, I agree it will be a low-priority issue, thanks.

best wishes.