Bihan Rana
Bihan Rana
> > This response does not provide location, but we don't need it because user has not supplied region argument and we only need the cheapest offer. The region field...
> @Bihan But what about the option C I suggested above? @peterschmidt85 What if Runpod changes its catalog before the trigger happens?
> This issue is stale because it has been open for 30 days with no activity. @peterschmidt85 The solution is to implement Runpod as as online provider. However to implement...
> Currently dstack uses gpuhunt runpod catalog collected daily. It includes only the offers available at the time of catalog generation. Since runpod availability changes throughout the day, some offer...
> This issue is stale because it has been open for 30 days with no activity. @peterschmidt85 Will I start implementing this feature?
Hey @deep-diver, 1. I’ve created a sample [README.md](https://github.com/Bihan/dstack/blob/nim_example/examples/deployment/nim/README.md) that follows dstack's structure. Please note that it assumes the reader is already familiar with NIM. Feel free to update it accordingly....
> @deep-diver, @Bihan what's the status of this PR? Should we merge it? @r4victor I will adjust the content till tomorrow morning and let you know.
> Any chance you could try `docker pull ghcr.io/huggingface/text-generation-inference:latest-rocm`? ROCm FP8 support was improved yesterday: > > #2588 @danieldk Yes sure.
@danieldk Deployed TGI with `neuralmagic/Meta-Llama-3-70B-Instruct-FP8` and it worked.
> @Bihan please comment if Ollama works or not on AMD and TPU @Ayush9026 If Ollama doesn't work with AMD or TPU, this should be stated (and ideally with the...