data-on-eks icon indicating copy to clipboard operation
data-on-eks copied to clipboard
verified publisher icon awslabs

[Inference]: Mistral7b on GPUs with JARK stack with Ray Serve

Open vara-bonthu opened this issue 1 year ago • 3 comments

vara-bonthu avatar Apr 08 '24 15:04 vara-bonthu

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

github-actions[bot] avatar May 16 '24 00:05 github-actions[bot]

I am working on this issue, can you please assign this roadmap item to me.

sheetaljoshi avatar Jul 03 '24 16:07 sheetaljoshi

There's an issue with using the HF Transformer versions with mistral-7b-intsruct-v0.2 model. https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/discussions/148

ratnopam avatar Jul 23 '24 20:07 ratnopam