Resume not working on Azure Batch

[alliemclean]

Bug report

resume option works locally but not with azure batch. I've gone through all the troubleshooting steps and can't find the reason the cache is invalidated

Expected behavior and actual behavior

resume should use the cache and not re-process cached jobs

Steps to reproduce the problem

link to input

nextflow run pipeline_test.nf \
 --name "resume_test_azure" \
 --in az://pipeline/input/GCA_000001215.4_current_ids.vcf.gz \
 --lines 10000 \
 --outdir az://pipeline/resume_results/ \
 -w az://pipeline/resume_working/ \
 -c azure.config \
 -dump-hashes \
 -resume

Program output

Uploaded logs for local and batch runs, run command lines, and config files: test_resume.tar.gz

• nextflow.log - the azure batch run that should have resumed and did not (-dump-hashes was used)
• nextflow.log.1.log - the original azure batch run (-dump-hashes was used)
• nextflow.log.2.log - the local run which resumed correctly
• nextflow.log.3.log - the original local run

Environment

• Nextflow version: 22.10.6.5843
• Java version: 17.0.6 2023-01-17 LTS
• Operating system: linux
• Bash version: GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
• azure plugin: [email protected]

Additional context

resume works with local execution and not with azure. Cache hashes are all the same as far as I can tell.

resume is also working on batch (same account, same settings) with a different pipeline.