Huang Xin

Environment: 6xA6000 48GB with Ubuntu 22.04, Pytorch 1.13.0 I ran into a generation problem after following your instruction to convert LLaMA-7B weight using your attached script. I simply used the...

Support LoRA training

5

Is there a plan to support PEFT methods like LoRA training in maxtext to support larger model fine-tuning / continue pretraining so that bigger models like LLaMA-3-70B can be trainined...

vmem OOM on TPU

1

Hi, I tried to run your script on Cloud TPU v4-64, but failed with following error: `jaxlib.xla_extension.XlaRuntimeError: RESOURCE_EXHAUSTED: XLA:TPU compile permanent error. Ran out of memory in memory space vmem....

Add a conversion script from maxtext gemma-2 and gemma-3 to huggingface format

2

# Description Create a conversion script from maxtext gemma-2 including gemma-2 2b/9b/27b to huggingface format. # Tests I have converted the maxtext version of gemma-2-9b-it and gemma-2-27b-it into huggingface format...