Jay Desai comments

Results 45 comments of


                                            Jay Desai

RuntimeError: 'weight' must be 2-D while training Flan-T5 models with stage 3

Actually, it comes from using bertScore ``` def evaluate(data_loader, model,tokenizer,print_samples=False,metric='bertscore_simple',device=None,**kwargs): """ Compute scores given the predictions and gold labels """ if device is not None: model = model.to(device) inputs,outputs, targets...

RuntimeError: 'weight' must be 2-D while training Flan-T5 models with stage 3

The error you mentioned at https://github.com/huggingface/peft/issues/168 is about both training and inference. Do you still have the errors? YES Your latest error is in computing metrics. Do you have no...

Any plans on writing a paper?

Will do, thanks

Int8 training example error with multi-gpu - flan-t5

![temp](https://user-images.githubusercontent.com/12378820/226692535-faf3d6a4-0431-4590-8381-9162d225d30e.jpg) Rename to temp.ipynb .

Int8 training example error with multi-gpu - flan-t5

To reproduce, option 2 is to comment out os.environ['CUDA_VISIBLE_DEVICES']='0' line in this example https://github.com/huggingface/peft/blob/main/examples/int8_training/Finetune_flan_t5_large_bnb_peft.ipynb and run it on multi-gpu instance.

Int8 training example error with multi-gpu - flan-t5

No I didn't

Model loaded from_pretrained Lora model goes OOM when trained.

@dukesun99 @pacman100 is there something like inference_mode that has to be turned off or anything missing here which is causing this behavior ?

Model loaded from_pretrained Lora model goes OOM when trained.

will try it out, thanks

Use Cerebras-GPT for fine tuning.

Did you guys start training cerebras gpt? any checkpoints that you can share ?

torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError: ========================================================= ./run.py FAILED --------------------------------------------------------- Failures: --------------------------------------------------------- Root Cause (first observed failure): [0]: time : 2023-09-22_18:11:04 host : xxx rank : 0 (local_rank: 0) exitcode : -11 (pid:...