jocastrocUnal

Loss drop in cero

15

Hi, i'm training "TheBlock/guanaco-7b-hf" with qlora. i have almost the same script from colab tutorial ```python import os os.environ["CUDA_VISIBLE_DEVICES"]="0" import torch import wandb from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig model_id...

⚠️INSTRUCTIONS: - Enter ONE "x" inside the brackets [x] to choose the answer - [x] Example - [ ] Example2 **Have you already searched for your ISSUE among the resolved...