Haihao Shen comments

Results 22 comments of


                                            Haihao Shen

[NeuralChat] Support user management in backend server

Let's take a look how vLLM and TGI supports. we can leverage them and add what we need if missing.

422 Unprocessable Entity using Neural Chat via OpenAI interface with meta--lama/llama-2-7b-chat-hf

Thanks @brent-elliott for reporting the issue. @huiyan2021 @lvliang-intel please take a look.

weight quantization

You can find "scale_params" in the quantized prototxt

[RFC] options about low-bit GEMM kernels contribution on x86 CPUs

Thanks @casper-hansen. We briefly discussed this on X about adding CPU optimizations to AutoAWQ, and we are going to create a PR soon as you suggested.

ChantQnA example on Xeon in AWS needs security groups to be opened

Done in https://github.com/opea-project/GenAIExamples/pull/156

Involvement with the CNCF Cloud Native AI Working Group

This sounds interesting! Thanks @raravena80 for bringing up. @mkbhanda

Remove Intel copyright from sourcecode

Hi @arun-gupta, Thanks for reporting the issue. WIP to check with legal about the change and the potential impact. Btw, could you please point to the recommended one by LF...

Add new DocIndexRetriever example

> > @xuechendi It seems the new GenAI example could be part of ChatQnA (RAG). Does it make sense to improve ChatQnA to add this example? > > @hshen14 ,...

[RFC]: Dynamic pipeline composition

> > OPEA already has [GenAI Microservices Connector](https://github.com/opea-project/GenAIInfra/tree/main/microservices-connector) for composing pipeline dynamically, please check via https://github.com/opea-project/GenAIInfra/blob/673e7c820667c92f56460161a20c054803cb431f/microservices-connector/README.md for some introduction. > > This is not the dynamic pipeline this RFC targets....

Not able to build ChatQnA on Mac

Mac is not yet in OPEA support list. It would be super interesting to add Mac support in OPEA roadmap.