nextflow
nextflow copied to clipboard
Fix PBSPRO queue status fetch
Closes #5528
A simple conditional if statement to check if the output is empty fixes the issue.
Deploy Preview for nextflow-docs-staging canceled.
| Name | Link |
|---|---|
| Latest commit | 0471dff3967a2e99e6c11bde15f6eb433822cc37 |
| Latest deploy log | https://app.netlify.com/sites/nextflow-docs-staging/deploys/673f133437a460000820fa6d |
I have a sneaking suspicion that this worked on whatever PBSPro cluster it was tested on, and maybe the status text changed across different versions. It would be good to have some clear documentation on how the status text is supposed to look, so that we can test against it, like here: https://github.com/nextflow-io/nextflow/blob/fa0e8e0f3d01e27dac60230d304dcec95dd1c2f5/modules/nextflow/src/test/groovy/nextflow/executor/PbsProExecutorTest.groovy#L219-L238
At the same time, I'm pretty sure the PBSPro executor was added at my request, to support my university cluster 😅 And I do remember seeing these "failed to fetch queue status" warnings but I never figured out why they were happening
Hello @bentsherman,
You are correct—I’ve been encountering this issue on Imperial’s cluster. It seems that qstat -B returns a phantom empty server string, which causes an error when passed to qstat -f. I have detailed the output in the mentioned issue.
The job state parsing portion of the code, however, appears to work perfectly and functions as intended. The failure lies in fetching the job state strings.
With my fork, qstat -f $( qstat -B | grep -E -v '(^Server|^---)' | awk -v ORS=' ' '{if ($1 != "") print "@"$1}' ) | { grep -E '(Job Id:|job_state =)' || true; } outputs:
Job Id: 297959.pbs
job_state = Q
Which is exactly what we want. My queued job is labelled appropriately.
For current version, qstat -f $( qstat -B | grep -E -v '(^Server|^---)' | awk -v ORS=' ' '{print "@"$1}' ) | { grep -E '(Job Id:|job_state =)' || true; } outputs:
qstat: illegally formed destination: @