llama-recipes

llama-recipes copied to clipboard

## Main Goal Building an e2e recipe for building chatbots. ### High level idea We want to focus on following stages: - Data pipelines for creating datasets for chatbots -...

HamidShojanazeri

peft_method works fine with lora, but pops error when using prefix and llama_adapter

10

### System Info [pip3] numpy==1.26.3 [pip3] torch==2.2.0+cu118 [pip3] triton==2.2.0 [conda] numpy 1.26.3 pypi_0 pypi [conda] torch 2.2.0+cu118 pypi_0 pypi [conda] triton 2.2.0 pypi_0 pypi GCP instance g2-standard-48, 4xL4 GPU ###...

cyberyu

evaluation

2

### System Info python eval.py --model hf --model_args pretrained=/mnt/sdb/zjk/llama2/llama-recipes/Llama-2-7b-hf,dtype="float",peft=/mnt/sdb/zjk/llama2/llama2-lora --tasks hellaswag --num_fewshot 10 --device cuda:0 --batch_size 8 error： 2024-04-02:06:58:12,156 ERROR [eval.py:226] An error occurred during evaluation: module 'lm_eval.tasks' has no...

ZHANGJINKUI

### System Info 路径错误 ### Information - [X] The official example scripts - [ ] My own modified scripts ### 🐛 Describe the bug from lm_eval.utils import make_table改为from lm_eval.evaluator import...

ZHANGJINKUI

# What does this PR do? - Adds a note on using the correct CUDA version for pip installs. - Removes 'test' from the pytorch whl install

subramen

# What does this PR do? Remove openai from example notebook and llm.py class. Simplify notebook layout and highlight additional packages that need to be installed. As discussed with @HamidShojanazeri...

tryrobbo

Implement H2O for long context inference on summarization tasks

2

This is add the implementation of H2O algorithm for efficient long context inference of Llama models. Current implementations are based on the Huggingface transformers and tests on summarization tasks, including...

Kyriection

BUG2

2

### 🚀 The feature, motivation and pitch I propose adding a feature to improve the visualization of loss curves in LLaMA-Recipes. Currently, the loss curves may exhibit spikes or irregularities,...

Phani1609

### System Info PyTorch version:2.2.1 ### Information - [X] The official example scripts - [X] My own modified scripts ### 🐛 Describe the bug The pretraining loss curve in LLaMA2...

Phani1609

Analysis of loss spikes in LLaMA pretrain

1

Dear LLaMA Teams, A huge thank you for making your remarkable work available to the public! I've taken a close look at the pretraining loss curves depicted in Figure 1...

zhipeng93