codetomakechange

@muellerzr > @codetomakechange out of curiosity have you found a way to do this _without_ accelerate launch? (aka native torch-xla?) If not that's okay, will look into this soon I...

> Probably, you can't do this because the TPUs from different VMs are not connected. If you want greater compute power, you need to use VMs with higher TPU cores...