qlora icon indicating copy to clipboard operation
qlora copied to clipboard

unable to run the training code

Open sathish4ds opened this issue 1 year ago • 3 comments

When i run the training code in colab its running when I try to run in local gpu it is asking for wandb api key when i run import transformers

needed for gpt-neo-x tokenizer

tokenizer.pad_token = tokenizer.eos_token

trainer = transformers.Trainer( model=model, train_dataset=data["train"], args=transformers.TrainingArguments( per_device_train_batch_size=1, gradient_accumulation_steps=4, warmup_steps=2, max_steps=10, learning_rate=2e-4, fp16=True, logging_steps=1, output_dir="outputs", optim="paged_adamw_8bit" ), data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False), ) model.config.use_cache = False # silence the warnings. Please re-enable for inference! trainer.train()

can anyone please give a advice to run this code, please find the notebook error below

Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving. wandb: Logging into wandb.ai. (Learn how to deploy a W&B server locally: https://wandb.me/wandb-server) wandb: You can find your API key in your browser here: https://wandb.ai/authorize wandb: Paste an API key from your profile and hit enter, or press ctrl+c to quit:

sathish4ds avatar Jun 14 '23 06:06 sathish4ds

It's too much trouble to run on a local GPU. Much faster to do it on colab pro if that is an available option.

Preetika764 avatar Jun 14 '23 08:06 Preetika764

My data is confidential data so want to setup in on prim can any one help me in on prim code

sathish4ds avatar Jun 15 '23 04:06 sathish4ds

just look up how to setup wandb @sathish4ds https://docs.wandb.ai/quickstart or disable it

JulesGM avatar Jun 15 '23 20:06 JulesGM