llm-foundry
llm-foundry copied to clipboard
mosaicml
LLM training code for Databricks foundation models
checkpoints continuous training, when re-saving, this error occurs 
## ❓ Question Does the inference command have a public api endpoint or batching of requests? ## Additional context I was wondering if this could be deployed on a production...
## 🚀 Feature Request The script located under scripts/inference for converting HF checkpoint to FT format doesn't work for MPT-7B-Storywriter because it has clip_qkv = 6 unlike other MPT-7B models...
## ❓ Question Hi, I am looking for the matric to compare the inference speed of the 7B, 13B and 70B models. More precisely I am looking for something like...
## 🚀 Feature Request I found there is only ICL Benchmark in eval folder, but the HumanEval Benchmark is reported in MPT-30B. I want to reproduce the HumanEval results with...
## ❓ Question Would you please add the hardware configuration details like Network bandwidth etc. in your benchmark page [here](https://github.com/mosaicml/llm-foundry/tree/main/scripts/train/benchmarking). ## Additional context
## ❓ Question Is the HF inference speed the same as MosaicML API inference? (regarding starter and enterprise inference solutions)
It is quite interesting that there are chat and instruct models on the same structure! Why can’t we train one model that can do both? Is that because 13B is...
## Environment ``` PyTorch version: 2.0.1+cu118 Is debug build: False CUDA used to build PyTorch: 11.8 ROCM used to build PyTorch: N/A OS: Debian GNU/Linux 11 (bullseye) (x86_64) GCC version:...
## Environment --------------------------------- System Environment Report Created: 2023-06-24 01:07:08 UTC --------------------------------- PyTorch information ------------------- PyTorch version: 2.0.1 Is debug build: False CUDA used to build PyTorch: 11.8 ROCM used to...
