intel-extension-for-transformers icon indicating copy to clipboard operation
intel-extension-for-transformers copied to clipboard

system prompt can't be assigned via neuralchat frontend

Open redhairerINTEL opened this issue 1 year ago • 5 comments

neuralchat already synced RESTful API with latest OpenAI protocol via 2e1c79d9b99db8bc004d67235fc6df51ca1d238e But neuralchat frontend don't have field to assign system prompt.

backend log

INFO:     127.0.0.1:58004 - "POST /v1/chat/completions HTTP/1.1" 200 OK
2024-02-23 04:14:29,602 - root - INFO - [INST] hi [/INST]
Batches: 100%|██████████| 1/1 [00:00<00:00, 86.64it/s]
2024-02-23 04:14:29,620 - root - INFO - Chat with QA Agent.

frontend log

2024-02-23 04:14:29 | INFO | gradio_web_server | ==== request ====
{'model': '/root/intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/examples/deployment/textbot/backend/Llama-2-7b-chat-hf/', 'messages': [{'role': 'user', 'content': 'hi'}], 'temperature': 0.001, 'top_p': 0.95, 'max_tokens': 512, 'stream': True}
2024-02-23 04:14:32 | INFO | httpx | HTTP Request: POST http://127.0.0.1:8080/v1/chat/completions "HTTP/1.1 200 OK"
image

redhairerINTEL avatar Feb 23 '24 09:02 redhairerINTEL

System prompt can be assigned via neuralchat backend now. The frontend feature will be completed by @WenjiaoYue later because of schedule of tasks.

letonghan avatar Feb 28 '24 02:02 letonghan

There are more urgent tasks now. Once completed, this part of the function will be added.

WenjiaoYue avatar Feb 28 '24 02:02 WenjiaoYue

Hi, we are now ready to start developing this part of the feature. Could you please provide the link to the frontend code you are referring to? @redhairerINTEL

WenjiaoYue avatar Mar 11 '24 08:03 WenjiaoYue

this is what I used. https://github.com/intel/intel-extension-for-transformers/tree/main/intel_extension_for_transformers/neural_chat/ui/gradio/basic

redhairerINTEL avatar Mar 18 '24 04:03 redhairerINTEL

Thank you very much. We will add this feature and will inform you promptly of any updates.

WenjiaoYue avatar Mar 20 '24 02:03 WenjiaoYue

Use API to assign system prompt directly. close it.

redhairerINTEL avatar May 07 '24 09:05 redhairerINTEL