openvino_notebooks
openvino_notebooks copied to clipboard
openvinotoolkit
Does nncf support qwen-1.8B or phi-2 ?
Describe the bug
Hello, I came across some usage instructions for the qwen model in Jupyter Notebook(254-llm-chatbot), but it seems that there is no specific guide for using qwen 1.8B. I attempted to modify the corresponding code to work with the 1.8B version, but it doesn't seem to run successfully. Could you please advise me on what steps I should take? Thank you.
Installation instructions (Please mark the checkbox) [ yes ] I followed the installation guide at https://github.com/openvinotoolkit/openvino_notebooks#-installation-guide to install the notebooks.
Additional context
qwen/Qwen-1_8B-Chat
It seems that there are some issues with quantization accuracy loss occurring.
Hi @sanbuphy Thanks for your feedback
How about the accuracy of FP16 version ?
Hi @sanbuphy Thanks for your feedback
How about the accuracy of FP16 version ?
Ok, Let me check
Hi @sanbuphy Thanks for your feedback
How about the accuracy of FP16 version ? @OpenVINO-dev-contest hi i meet the same error:
@sanbuphy regarding ph-2 support, unfortunatly, this model is not good in chatbot scenario as it is not fine-tuned for this case. We prepared separated demo, that allow you run it as text generator for answering questions or following instructions: https://github.com/openvinotoolkit/openvino_notebooks/pull/1577 The only limitation that our model support is based on model implementation from transformers lib and original phi-2 model is not migrated on it yet (hope migration happens soon based on discussions on hf hub)
@sanbuphy regarding ph-2 support, unfortunatly, this model is not good in chatbot scenario as it is not fine-tuned for this case. We prepared separated demo, that allow you run it as text generator for answering questions or following instructions: #1577 The only limitation that our model support is based on model implementation from transformers lib and original phi-2 model is not migrated on it yet (hope migration happens soon based on discussions on hf hub)
Thank you ! get it ,so, I just need to wait for qwen to fix the issues, as it seems there are still some bugs present.
qwen/Qwen-1_8B-Chat
Hi I think Qwen-7b-chat can work now, and we are trying to enable Qwen-1_8B-Chat. We will keep you in tune. thanks
qwen/Qwen-1_8B-Chat
Hi I think Qwen-7b-chat can work now, and we are trying to enable Qwen-1_8B-Chat. We will keep you in tune. thanks
Does Openbino currently support the Qwen-14B-Chat model?Or baichuan-13B-Chat model, because 7B or 6B models are still too weak
LLM notebook is updated with QWEN1.5-1.8b, pls check: https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot
LLM notebook is updated with QWEN1.5-1.8b, pls check: https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot
Thanks !
LLM notebook is updated with QWEN1.5-1.8b, pls check: https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot
i want to ask if openvino support RAG rerank model now? like : https://huggingface.co/maidalun1020/bce-reranker-base_v1
LLM notebook is updated with QWEN1.5-1.8b, pls check: https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot
i want to ask if openvino support RAG rerank model now? like : https://huggingface.co/maidalun1020/bce-reranker-base_v1
we have new RAG notebook, which can support rerank model: https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-rag-langchain
LLM notebook is updated with QWEN1.5-1.8b, pls check: https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot
i want to ask if openvino support RAG rerank model now? like : https://huggingface.co/maidalun1020/bce-reranker-base_v1
we have new RAG notebook, which can support rerank model: https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-rag-langchain
cool！ let me try it