Richard Sun issues

Results 5 issues of


                                            Richard Sun

Are there new examples of applications on glove-python?

Hi, I am new to NLP and interested to explore the hype of word2vec. I wanna carry out some intrinsic evaluation such as "man-women=father-mother". In gensim package, we can do...

How to migrate the finetune/full.py to a multi-node and multi-gpu environment?

I would like to make use more than 1 of the servers in my lab. Is it convenient to migrate the fully fine-tuning code to a multi-node environment? Has anybody...

OOM when fully finetuning llama-7B using the deepspeed strategy

I tried to fully finetune llama-7b with deepspeed based on the code https://github.com/Lightning-AI/lit-llama/blob/main/finetune/full.py. I replaced the FSDPStrategy with DeepSpeedStrategy(offload_optimizer=True, offload_parameters=False, pin_memory=True, offload_optimizer_device='cpu'), but there would always be an OOM error...

Error when Resuming pre-training

I ran into this error when resuming the pre-training on the pre-trained weights: "ImportError: cannot import name '_FaultTolerantMode' from 'pytorch_lightning.utilities.enums' " Has anyone seen this issue before? Thanks! I modified...

how can we add the conversation templates for new LLMs such as Llama3 and Qwen2?

how can we add the conversation templates for new LLMs such as Llama3 and Qwen2? Thanks!