llama-recipes

llama-recipes copied to clipboard

Finetune CodeLlama-7b-Instruct-hf on private dataset

3

I hope this message finds you well. I recently had the opportunity to experiment with the Codellama-7b-Instruct model from GitHub repository and was pleased to observe its promising performance. Encouraged...

HumzaSami00

Wrong truncation of training examples in alpaca dataset

2

### System Info PyTorch version: 2.0.1+cu117 Is debug build: False CUDA used to build PyTorch: 11.7 ROCM used to build PyTorch: N/A OS: Red Hat Enterprise Linux release 8.8 (Ootpa)...

YosiMass

Not sure I have FSDP working properly; any insight on fine-tuning VRAM req's?

4

Sorry for the stupid question, I'm not sure why I can't seem to investigate this well, and it's driving me nuts. I've tried running the example finetuning script on the...

shaded-blue

Tensorboard/Wandb

3

### 🚀 The feature, motivation and pitch Hi all! Has anyone edited the code so to support logging on Tensorboard or WanDB? Thanks! ### Alternatives _No response_ ### Additional context...

zepmck

NotImplementedError: Cannot copy out of meta tensor; no data!

2

### System Info 2.0.1+cu118, driver 535.86.05 ### Information - [X] The official example scripts - [ ] My own modified scripts ### 🐛 Describe the bug When you run on...

timothylimyl

a question about video memory occupancy

1

### 🚀 The feature, motivation and pitch i follow the instructions like python llama_finetuning.py \ --use_peft \ --quantization \ --model_name "meta-llama/Llama-2-7b-chat-hf" \ --output_dir Path/to/save/PEFT/model i wander if i do not...

191220042

undefined symbol: cget_col_row_stats

3

### System Info ``` torch=2.0.1+cu118 NVIDIA TITAN RTX 3090 NVIDIA-SMI 525.116.04 Driver Version: 525.116.04 CUDA Version: 12.0 ``` ### Information - [X] The official example scripts - [ ] My...

cdhx

convert_llama_weights_to_hf.py json.decoder.JSONDecodeError

3

### System Info colab GPU：v100 ### Information - [ ] The official example scripts - [ ] My own modified scripts ### 🐛 Describe the bug  ### Error logs...

munitioner

Questiones about fsdp + pure bf16

4

I want to continue pre-training the LLMa70b model in order to add Chinese tokens and train it on Chinese data. I'm considering using FSDP along with pure bf16. However, I...

kongjiellx

llama 2 distributed training on AWS Sagemaker

4

Hi, I am going to do distributed training of llama on aws sagemaker as managed training across multiple devices/nodes. Sagemaker provides data parallel and model parallel distributed training in sagemaker....

premanand09