Is each machine its own Regional managed instance group?

[carbocation]

I noticed that I hit a GCP quota "Regional managed instance groups" (N=1,250 in the region of interest) when launching a job with a few thousand tasks (dsub 0.5.0, using google-batch). Is it possible that dsub is creating one "Regional managed instance group" per machine?

[wnojopra]

Hi @carbocation

Thanks for reporting. Yes, it would appear dsub is creating one "Regional managed instance group" per task. But that wasn't intentional.

dsub creates one taskGroup per task, which followed the pattern we had done for the Lifesciences API (which did not touch managed instance groups). But it does seem each taskGroup created ends up created a managed instance group.

Its possible we could rework this so that dsub could set a single taskGroup with many tasks instead, but this would be something to look into.