Omead Pooladzandi

Pytorch XLA/PJRT TPU support

1

### Feature request Pytorch XLA/PJRT TPU support for bitsandbytes ### Motivation Would allow for faster and more memory efficient training of models on TPUs. ### Your contribution Happy to provide...

Hi quick question, Should we expect FFCV to work plug and play with TPUs if we are using PyTorch with XLA? Thank you, Omead

XLA Pod Request

2

Thank you for the repo. I am wondering if a recipe for TPU pods can be added. I have access to v4-32 and want to train a LLaMA model from...

I guess more of a general question. I want to train a 7B LLaMA 2 on some data from scratch. What datasets would you recommend? Best way to tokenize etc....

Issue Training on Google Colab

4

I am getting the following error when trying the train a model following the readme. Traceback (most recent call last): File "/usr/lib/python3.10/pydoc.py", line 443, in safeimport module = __import__(path) File...

Hi thanks for the great poisson. I was wondering if you can provide some code to attack a ViT on CIFAR10 or imagenet.

Hello, I'm trying to pre-train a llama model using fabric on a TPU Pod. I have access to a few v4-32s. training on a v4-8 is trivial using pytorch XLA...