llm-foundry

llm-foundry copied to clipboard

Hi, I could do with a pointer on what's going wrong here. I've follows instructions, and somehow ended up with Torch 1.13.1 when I think it needs 2.x Cheers, J....

jamesd256

Fused Cross Entropy is not installed. Either (1) have a CUDA-compatible GPU and `pip install .[gpu]`, or (2) set your config model.loss_fn=torch_crossentropy.

1

Initializing model... Traceback (most recent call last): File "/content/llm-foundry/llmfoundry/models/mpt/modeling_mpt.py", line 619, in __init__ from flash_attn.losses.cross_entropy import CrossEntropyLoss as FusedCrossEntropyLoss # type: ignore # isort: skip File "/usr/local/lib/python3.10/dist-packages/flash_attn/losses/cross_entropy.py", line 9, in...

stanlitoai

Reproduce result of Boolq on LLaMA-7B

4

Hi The __zeroshot__ performance on BoolQ in LLaMA paper is 76.5. While the llm-foundry only 62.16 (zero-shot) when following `tasks.yaml`. The result in blog is a few-shot ? How about...

mx8435

Add GGML support

4

It would be nice to have the model supported by GGML, so as to make quantized versions of it or future derivatives also run without GPU. See https://github.com/ggerganov/llama.cpp/issues/1333#issuecomment-1536725381 I gave...

jploski

getting OOM on 8 nvidia GPUs with 40GB memory each

12

using `scripts/train/train.py` yaml: yamls/mpt/finetune/7b_dolly_sft.yaml

arpitg1991

HuggingFace -> Hugging Face

eltociear

FasterTransformer

31

Hi, I saw in mpt model card that the models could run with FasterTransformer I didn't find any details about that anywhere can you guys share the conversion scripts or...

xgal

Finetuning

2

Hi Team, I tried the finetuning code given in repo with 7b_dolly_sft.yaml, I ran for one epoch. Please find the details below: [epoch=1][batch=927/927]: Train time/batch: 926 Train time/sample: 59238 Train...

canamika27

multilingual ability

9

how about the multilingual ability of MPT

yangjianxin1

Not an issue, a question - Peft/LoRa finetuning a possibility?

14

Another noob question... Is is possible to reduce the resource burden for fine tuning by using Peft/LoRa techniques? If not will it be possible in the future with MPT models?

jamesd256