Aflah

Added Helpful Information for Recreating Results

1

Added Possible Error Fixes

Adding GPT-NeoX

7

I followed along the instructions [here ](https://github.com/sgl-project/sglang/issues/81#issuecomment-1917086172) to add GPT-NeoX support which would bring support for the Pythia model family and other similar architecture models. Reference: https://github.com/sgl-project/sglang/issues/157#issue-2122338478 FIXED (Keeping Logs...

Slow weight loading

9

Whenever I try to load the Mixtral models it takes very long and at the end instead of actually starting the server I get a similar error as the one...

[Usage]: What's the minimum VRAM needed to use entire context length for Llama 3.1 70B and 405B

1

### Your current environment Libraries Installed - ``` "vllm==0.5.5", "torch==2.4.0", "transformers==4.44.2", "ray", "hf-transfer", "huggingface_hub" ``` ### How would you like to use vllm Hi I want to run Llama 3.1...

Expected Data Format

1

### ❓ The question I was looking at the config files and noticed that the config files sometimes point to `.npy` files for the dataset. Is there any script to...

### Feature Request Several models like Pythia, OLMo etc. use revisions to store different checkpoints. It would be great if supplying a revision arg is supported for these models. ###...

Pretraining an OLMo model on the SlimPajama dataset

16

Hi! I am planning to test pretraining OLMo 1B model on the slim pajama dataset. I was trying to follow the tutorial for tinyllama but one of the steps for...

[Example] OpenRLHF Integration

3

This PR adds examples for running DPO, SFT and RM Training using OpenRLHF on SkyPilot. Verified the training runs on GCP