configs
configs copied to clipboard
Add Azure specifc Sarek pipeline config
Currently this is part of the Azure specific config for Sarek, but can be optimized.
azure {
batch {
pools {
auto {
autoScale = true
sku = "batch.node.centos 7"
offer = "centos-container"
publisher = "microsoft-azure-batch"
vmType = 'Standard_E64_v3'
vmCount = 1
}
}
}
}
What would the benefit of having a Sarek specific Azure config be over updating the general Azure Batch config?
Agreeing with @adamrtalbot not sure why this has to be sarek specific.
@maxulysse @adamrtalbot the pipeline specific config will have the vm_types that are a recomendation to scale for cost and time on azure for specific pipelies as the default settings don't translate well for azure currently.
Given Azure VM sizes are region, account and time specific, it would make more sense to make it a parameter you can set. We don't have any E64s in our quota and I do not have the budget to increase that!
params {
azure_vm_type = "Standard_E64_v3"
}
azure {
batch {
pools {
auto {
vmType = params.azure_vm_type
}
}
}
}
I agree. I think a parameter that can set different vm_types with a default if available.
Thanks, Venkat
I have something that is already pretty close, if I open a PR could you give it some testing for me?
@adamrtalbot yes I can do that. Are you thinking pipeline specific or generic to all pipelines?
@adamrtalbot yes I can do that. Are you thinking pipeline specific or generic to all pipelines?
Take a look here: https://github.com/nf-core/configs/pull/406
Woops, I pushed to this repo instead of my fork. Didn't realise I had permission to do that. Hopefully that's OK.
Woops, I pushed to this repo instead of my fork. Didn't realise I had permission to do that. Hopefully that's OK.
I think we have protection on the main branch, so you can push on a different one. But could be worth checking this out @nf-core/core
Yeah, master
should be protected..
@adamrtalbot do you think we can close this one?
Yes. I believe @vsmalladi has an even better version?
Yes will add one soon