fix: Cap azcopy memory usage at 80% of container memory

[adamrtalbot]

azcopy was consuming more memory than the hard cap of the container memory. This PR does two things:

• Downloads the files in linear, since azcopy is super fast anyway
• Caps it to 80% of the container memory, which should be sufficient for most cases while preventing excessive memory usage.

Fixes #6161

[netlify[bot]]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	07fe8223bdd38188ab2667fc96132a60cbcc6175
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs-staging/deploys/68432dd19a56fd0008a773bc

[adamrtalbot]

Doesn't work yet

[adamrtalbot]

Pipeline to test:

process create_file {
    machineType "Standard_D16D_v5"
    memory 1.GB
    container "ubuntu:23.10"

    input:
        val n

    output:
        path "test.file", emit: "tmpFile"

    script:
    """
    fallocate -l 50G test.file
    """
}

process do_sth {
    machineType "Standard_D16d_v5"
    memory 1.GB
    container "ubuntu:23.10"

    input:
    path "???"

    output:
    path "out.txt"

    script:
    """
    ls > out.txt
    """
}

workflow {
    n_ch = Channel.of(1..3)
    create_file(n_ch)
    do_sth(create_file.out.collect())
}

[adamrtalbot]

Does not fix the problem again

[adamrtalbot]

THIS TIME.