Simon Mo
Simon Mo
hostPath a possible workaround.
Nice! I think we need to add a test here. @DarkLight1337 can you suggest a good place to test the model?
Contribution welcomed! https://docs.vllm.ai/en/latest/models/adding_model.html
It's supported in main branch, which you can use when building from source. It will be in next release (ETA tomorrow)
Reopening this to focus on simple offline use case.
Yes. We will integrate it.
@mmoskal please help review! thank you
I'm ok with 88 as it helps readability and 80 is a bit out dated... As long as we don't create issue synching upstream for all the PRs I'm in...
Interestingly Triton recently supported macOS with Apple Silicon: https://github.com/triton-lang/triton/issues/3443
Good point. PR welcomed!