intel-extension-for-transformers
intel-extension-for-transformers copied to clipboard
system prompt can't be assigned via neuralchat frontend
neuralchat already synced RESTful API with latest OpenAI protocol via 2e1c79d9b99db8bc004d67235fc6df51ca1d238e But neuralchat frontend don't have field to assign system prompt.
backend log
INFO: 127.0.0.1:58004 - "POST /v1/chat/completions HTTP/1.1" 200 OK
2024-02-23 04:14:29,602 - root - INFO - [INST] hi [/INST]
Batches: 100%|██████████| 1/1 [00:00<00:00, 86.64it/s]
2024-02-23 04:14:29,620 - root - INFO - Chat with QA Agent.
frontend log
2024-02-23 04:14:29 | INFO | gradio_web_server | ==== request ====
{'model': '/root/intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/examples/deployment/textbot/backend/Llama-2-7b-chat-hf/', 'messages': [{'role': 'user', 'content': 'hi'}], 'temperature': 0.001, 'top_p': 0.95, 'max_tokens': 512, 'stream': True}
2024-02-23 04:14:32 | INFO | httpx | HTTP Request: POST http://127.0.0.1:8080/v1/chat/completions "HTTP/1.1 200 OK"
System prompt can be assigned via neuralchat backend now. The frontend feature will be completed by @WenjiaoYue later because of schedule of tasks.
There are more urgent tasks now. Once completed, this part of the function will be added.
Hi, we are now ready to start developing this part of the feature. Could you please provide the link to the frontend code you are referring to? @redhairerINTEL
this is what I used. https://github.com/intel/intel-extension-for-transformers/tree/main/intel_extension_for_transformers/neural_chat/ui/gradio/basic
Thank you very much. We will add this feature and will inform you promptly of any updates.
Use API to assign system prompt directly. close it.