Muhammad Ahtesham Ul Haq

if you are not generating the data and using the data provided in repo you do not need openai. train.py does not use openai package in any case.

I did not find the solution. but if someone wants to download the weights. following link has all the versions. https://huggingface.co/elinas

Can you tell me what GPUs you are using? try this: change this with --bf16 True with --fp16 True and hopefully it will work.

tf32 is supported by A100 and you are using V100

@yysjasmine did the command work with 4 A100 ?

> pip install git+https://github.com/zphang/transformers.git@llama_push > > Each version is as follows： numpy==1.24.2 rouge-score==0.1.2 fire==0.5.0 openai==0.27.2 sentencepiece==0.1.97 wandb==0.14.0 Have you used this for your environment?

torch.distributed.init_process_group(backend='nccl', init_method='env://', timeout=datetime.timedelta(seconds=1800))

No I have not yet. I did not find any solution. Now I am looking to have more GPUs to train it. regarding usage of saved model. seams like i...

set the environment variable. probably thats the variable OPENAI_ORG In linux you can do following export OPENAI_ORG="your-api-key"

Faced the same error. I was able to resolve it using model().cuda().half() But when I tested the model the results i got was something like this https://user-images.githubusercontent.com/88507331/230518491-741b0f32-de9d-433c-ba6f-8d85085d7578.png Not sure if...