Ratnopam Charabarti comments

Results 19 comments of


                                            Ratnopam Charabarti

Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2

This PR references an image that's submitted in https://github.com/ray-project/images/pull/18

Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2

Is anyone able to review this PR and merge if ok? Thanks.

Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2

Generated doc can be viewed at https://anyscale-ray--43046.com.readthedocs.build/en/43046/serve/tutorials/index.html.

[Serve] Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2

You can assign this issue to me.

[Serve] Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2

@GeneDer , I've raised a PR https://github.com/ray-project/ray/issues/43018 to store the output image. The github url of the image is referenced in the docs. Can you please merge the above PR?...

[Serve] Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2

@GeneDer, Thanks for looking into this. I raised https://github.com/ray-project/ray/issues/43018 for storing the output image. And in https://github.com/ray-project/ray/pull/43046, I've referenced that image URL in the doc. I checked the failures; looks...

[Serve] Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2

@GeneDer thanks a lot for your review. All checks are passing now.

[Serve] Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2

@GeneDer, thanks. I resubmitted for review.

Problem following tutorial Run a Replicated Stateful Application

Following this https://v1-24.docs.kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/#simulate-pod-and-node-downtime example and the readiness probe fails but the pod keeps running. Can anyone fix the doc or suggest a work-around? Update - Changing the command for introducing...

[Inference]: Mistral7b on GPUs with JARK stack with Ray Serve

There's an issue with using the HF Transformer versions with mistral-7b-intsruct-v0.2 model. https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/discussions/148