Jie

Random Training

12

Hi authors, In the SFTTrainer, we set "seed = 3407". But I find the training procedure is still random. the performance of test dataset and the change of loss are...

2 warnings: MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization") / kernel version 4.18.0

1

anaconda3/envs/alpaca-lora/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py:316: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization") Detected kernel version 4.18.0, which is...

batch inference

1

Hi authors, I want to test the performance of the Mistral7B on the test dataset. Is it only possible to do single sample inference (with model. generate(...))? Are there any...