Hanna Yukhymenko

icon indicating a website link https://savelife.in.ua/en/donate-en/

icon company ETH Zurich/Kyiv Polytechnic Institute icon location Zaporizhzhia, Ukraine icon location Statistics MSc @ ETH Zurich

Results 2 issues of Hanna Yukhymenko

Fail to distribute training on GCP TPU Pod+Object of type device is not JSON serializable Error when saving the model

2 comment

Hi, I am trying to finetune an LLM on v4-64 GCP TPU Pod. However, when I launch the training script, it runs 8 times on each host (worker) separately and...

TPU

Distributed training on multi-host v4/v5 TPU Pods is too slow

7 comment

## ❓ Questions and Help Hi! We are trying to train Gemma-2-9B on v4-64 and v5-128 Pod as mentioned in [this comment](https://github.com/pytorch/xla/issues/7987#issuecomment-2352326629). We use FSDP+SPMD setup on torch XLA 2.4.0...