pegasus icon indicating copy to clipboard operation
pegasus copied to clipboard

[PM-1653] Set when_to_transfer_output = ON_SUCCESS

Open mayani opened this issue 1 year ago • 3 comments

the planner should automatically set when_to_transfer_output to ON_SUCCESS if the condor version on the submit node is 8.9.7 or higher

https://htcondor.readthedocs.io/en/latest/version-history/development-release-series-89.html?highlight=when_to_transfer_files

You may now specify that HTCondor only transfer files when the job succeeds (as defined by success_exit_code). Set when_to_transfer_files to ON_SUCCESS. When you do, HTCondor will transfer files only when the job exits (in the sense of ON_EXIT) with the specified success code. This is intended to prevent unsuccessful jobs from going on hold because they failed to produce the expected output (file(s)).

Reporter: @vahi Watchers: @papajim @rynge @ryantanaka @vahi

mayani avatar Dec 14 '24 07:12 mayani

Author: @vahi

As pointed by @rynge it seems it is a typo. they probably mean it to be when_to_transfer_output

https://htcondor.readthedocs.io/en/v8_9_7/users-manual/file-transfer.html?highlight=WHEN_TO_TRANSFER_OUTPUT

mayani avatar Dec 14 '24 18:12 mayani

Author: @vahi

Reply from todd miller @condor

Can you comment, if just detecting the submit node condor version is enough to use this feature

The starter must also be sufficiently recent.  Submitting a job with ON_SUCCESS set will cause a submit node which supports the feature to add a constraint to the job so that it won't run anywhere that isn't recent enough to support ON_SUCCESS, but that's probably not what you want. :/ . 

We will do this for 5.1 . in the meantime users can still override this via condor profile when_to_transfer_output if they want to use it

mayani avatar Dec 14 '24 18:12 mayani

need to test this more after making the change. pushing to 5.1.1 after internal discussion

vahi avatar May 05 '25 21:05 vahi