Litserve <> Haystack

[srini047]

Notebook

Serve Haystack pipelines using Litserve.

Reason

With its core reason to use being to serve AI/LLM models. I would like to showcase the capabilities of Haystack with Litserve.

Prework

Eg:

import litserve as ls
from haystack import Pipeline, PredefinedPipeline


class HaystackPredefinedLitAPI(ls.LitAPI):
    def setup(self):
        self.pipeline = Pipeline.from_template(PredefinedPipeline.GENERATIVE_QA)

    def decode_request(self, request):
        return request["query"]

    def predict(self, question):
        return self.pipeline.run({"prompt_builder": {"question": question}})

    def encode_response(self, output):
        return {"output": output}


if __name__ == "__main__":
    server = ls.LitServer(HaystackPredefinedLitAPI(), accelerator="auto")
    server.run(port=8000)

I would like to show how custom pipelines can be built and run.

[TuanaCelik]

@srini047 if you created an example/demo we would be happy to share it/showcase it

[srini047]

@srini047 if you created an example/demo we would be happy to share it/showcase it

Sure, I shall work on the code with demo.

[bilgeyucel]

Hi @srini047, do you have any demos/notebooks to share? I'm happy to help and review!

[srini047]

Hi @srini047, do you have any demos/notebooks to share? I'm happy to help and review!

Hi @bilgeyucel , I would work on the implementation. It was parked in my bucket. Will share soon. Thanks.

[srini047]

Hi @srini047, do you have any demos/notebooks to share? I'm happy to help and review!

Hi @bilgeyucel , I would work on the implementation. It was parked in my bucket. Will share soon. Thanks.

Hey @bilgeyucel PFA this repo: https://github.com/srini047/haystack-litserve I have showcased the implementation of using Haystack with Litserve. Would like to get your feedback on this. Moreover how would you like this to be added to the repo, I'm bit confused about the same.

Looking forward to hearing from you, thanks much.

[srini047]

@bilgeyucel Any updates on this. I see you self assigned this but please let me know if I can assist you. I am happy to help. I just want to know the right way to incorporate into the cookbooks repo. Thanks.