Bhargav Solanki
Bhargav Solanki
@Iven2132 I created a fastapi + uvicorn endpoint for now.
I had made this feature request here . https://github.com/vllm-project/vllm/discussions/4350
Thanks for doing this, I was trying to train the bf16, 9b gemma2 -it, load4bit false, like i trained for mistal7b instruct and so on, but for gemma i keep...
@danielhanchen , ``` Unsloth currently does not work on multi GPU setups - sadly we are a 2 brother team so enabling it will require much more work, so we...
@danielhanchen , following up on this.
@danielhanchen , yes, i did. It does nothing. As a pro active attempt, since commenting that line did not work, i tried disable. Which yielded similar results as shared above....