John Uiterwyk
John Uiterwyk
I have use case as well that involves using an externally hosted model that is self hosted and located within a private network (or more simply another use cases is...
thanks @dhrubo-os, i've raised an issue for that request specifically: https://github.com/opensearch-project/ml-commons/issues/2142
hi @ylwu-amzn, please consider that exposing a self hosted LLM server to the public internet via a internet facing load balancer is a not a workable solution given the security...
Thanks @dhrubo-os . My motivation raising the issue to enable private ip addresses is specifically driven by security and data control considerations.
hi @dhrubo-os, i was wondering if there is any progress on this. i would love to see this included in 2.13 as it looks like a very small change; This...
thanks @dhrubo-os, great to hear there is progress on this! Also just wanted to say thanks for all you and your teams hard work. This project is incredibly valuable and...
Hi, was this ever implemented; I looked through the linked issues, and it seems to be a circular reference, of "implemented over there"? This would be a great feature to...
Hi folks, it was great to discover the addition of the `UIAdapter`, it very much fits our needs. I think it is important we keep UIAdapter generalized and being faithful...
same here, this is a blocker for me
No, as the maximum max_lora_rank is 64, going higher than that throws an error. I have adapters with rank 128 and 256 for certain uses cases, and can not serve...