Huang Xin

icon indicating a website link http://www.hxdavid.com icon indicating an email [email protected]

icon company A*STAR icon location Singapore icon location Beauty doesn't just show up, you need to observe it.

Results 3 issues of Huang Xin

Generation problem after / before instruction fine-tuning

Environment: 6xA6000 48GB with Ubuntu 22.04, Pytorch 1.13.0 I ran into a generation problem after following your instruction to convert LLaMA-7B weight using your attached script. I simply used the...

Support LoRA training

2 comment

Is there a plan to support PEFT methods like LoRA training in maxtext to support larger model fine-tuning / continue pretraining so that bigger models like LLaMA-3-70B can be trainined...

feature request

vmem OOM on TPU

1 comment

Hi, I tried to run your script on Cloud TPU v4-64, but failed with following error: `jaxlib.xla_extension.XlaRuntimeError: RESOURCE_EXHAUSTED: XLA:TPU compile permanent error. Ran out of memory in memory space vmem....