cylc-flow icon indicating copy to clipboard operation
cylc-flow copied to clipboard

Logging: reduce duplication of jobs-submit cmd in scheduler log

Open MetRonnie opened this issue 2 months ago • 0 comments

Partially addresses https://github.com/cylc/cylc-flow/issues/3647

When investigating an issue, the repeated logging of a batch jobs-submit command was rather irritating and hampered looking through the scheduler log.

Repro

Have a workflow with lots of tasks that will be batch-submitted but will fail to submit.

# flow.cylc
[task parameters]
    x = 1..10
[scheduling]
    [[graph]]
        R1 = foo<x>
[runtime]
    [[foo<x>]]
        platform = borked
# global.cylc
[platforms]
    [[borked]]
        hosts = localhost
        install target = localhost
        job runner = sge

Logging diff:

 ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
     [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x01/01|1|
 ERROR - [1/foo_x01/01:preparing] submission failed
 INFO - [1/foo_x01/01:preparing] => submit-failed
 WARNING - [1/foo_x01/01:submit-failed] did not complete the required outputs:
     x  succeeded
-ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
-    [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x02/01|1|
 ERROR - [1/foo_x02/01:preparing] submission failed
 INFO - [1/foo_x02/01:preparing] => submit-failed
 WARNING - [1/foo_x02/01:submit-failed] did not complete the required outputs:
     x  succeeded
-ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
-    [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x03/01|1|
 ERROR - [1/foo_x03/01:preparing] submission failed
 INFO - [1/foo_x03/01:preparing] => submit-failed
 WARNING - [1/foo_x03/01:submit-failed] did not complete the required outputs:
     x  succeeded
-ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
-    [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x04/01|1|
 ERROR - [1/foo_x04/01:preparing] submission failed
 INFO - [1/foo_x04/01:preparing] => submit-failed
 WARNING - [1/foo_x04/01:submit-failed] did not complete the required outputs:
     x  succeeded
-ERROR - [jobs-submit cmd] cylc jobs-submit --path=/bin --path=/usr/bin --path=/usr/local/bin --path=/sbin --path=/usr/sbin --path=/usr/local/sbin -- '$HOME/cylc-run/diamond/log/job' 1/foo_x01/01 1/foo_x02/01 1/foo_x03/01 1/foo_x04/01 1/foo_x05/01 1/foo_x06/01 1/foo_x07/01 1/foo_x08/01 1/foo_x09/01 1/foo_x10/01
-    [jobs-submit ret_code] 1
-    [jobs-submit out] 2025-12-08T18:23:34Z|1/foo_x05/01|1|
 ERROR - [1/foo_x05/01:preparing] submission failed
 INFO - [1/foo_x05/01:preparing] => submit-failed
 WARNING - [1/foo_x05/01:submit-failed] did not complete the required outputs:
     x  succeeded
 ...

Check List

  • [x] I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • [x] Contains logically grouped changes (else tidy your branch by rebase).
  • [x] Does not contain off-topic changes (use other PRs for other changes).
  • [x] No dependency changes
  • [x] Tests are included
  • [x] No changelog needed as minor
  • [x] No docs needed
  • [x] If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

MetRonnie avatar Dec 08 '25 19:12 MetRonnie