lua-openai
lua-openai copied to clipboard
leafo
OpenAI API bindings for Lua
1. Document the way to set timeout. AFAIK the default is 60s, but this may not always be sufficient. 2. Response streaming can be time-consuming and in certain situations a...
Consider such (real) api response: ``` data: {"id":"chatcmpl-7ZZCd6bd5DwGT9UdoK9Ph4","model":"llama2-70b","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}],"usage":null} data: {"id":"chatcmpl-7ZZCd6bd5DwGT9UdoK9Ph4","model":"llama2-70b","choices":[{"index":0,"delta":{"content":" "},"finish_reason":null}]} data: {"id":"chatcmpl-7ZZCd6bd5DwGT9UdoK9Ph4","model":"llama2-70b","choices":[{"index":0,"delta":{"content":" Hello!"},"finish_reason":null}]} data: {"id":"chatcmpl-7ZZCd6bd5DwGT9UdoK9Ph4","model":"llama2-70b","choices":[],"usage":{"prompt_tokens":11,"total_tokens":15,"completion_tokens":4}} data: [DONE] ``` The response objects are missing `"object": "chat.completion.chunk"`, so they aren't passing...
(On top of #10) This PR is created to discuss possible `OpenAI\_request` refactoring. What's changed: - Collect chunks during streaming, and instead of bare text, return synthetic response object, similar...
Every message ends with two newlines Specs: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format Fix #7
And do not demand `object: "chat.completion.chunk"` within the chunk object to support not fully OpenAI-compliant services Close #5
This is rather a question. Is there a reason to unconditionally alter openai defaults?
``` Run busted [2](https://github.com/leafo/lua-openai/actions/runs/8728698639/job/23949076209?pr=8#step:6:2) busted [3](https://github.com/leafo/lua-openai/actions/runs/8728698639/job/23949076209?pr=8#step:6:3) shell: /usr/bin/bash -e {0} [4](https://github.com/leafo/lua-openai/actions/runs/8728698639/job/23949076209?pr=8#step:6:4) env: [5](https://github.com/leafo/lua-openai/actions/runs/8728698639/job/23949076209?pr=8#step:6:5) MSYS: winsymlinks:nativestrict [6](https://github.com/leafo/lua-openai/actions/runs/8728698639/job/23949076209?pr=8#step:6:6) LUA_PATH: ;;/home/runner/.luarocks/share/lua/5.4/?.lua;/home/runner/.luarocks/share/lua/5.4/?/init.lua;/home/runner/work/lua-openai/lua-openai/.luarocks/share/lua/5.4/?.lua;/home/runner/work/lua-openai/lua-openai/.luarocks/share/lua/5.4/?/init.lua [7](https://github.com/leafo/lua-openai/actions/runs/8728698639/job/23949076209?pr=8#step:6:7) LUA_CPATH: ;;/home/runner/.luarocks/lib/lua/5.4/?.so;/home/runner/work/lua-openai/lua-openai/.luarocks/lib/lua/5.4/?.so [8](https://github.com/leafo/lua-openai/actions/runs/8728698639/job/23949076209?pr=8#step:6:9) [9](https://github.com/leafo/lua-openai/actions/runs/8728698639/job/23949076209?pr=8#step:6:10) 0 successes / 0 failures / 0...
Specs: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format As you can see, the specifications are rigidly defined, eliminating the necessity to use LPEG and iterative trial-and-error JSON decoding. In addition, the current implementation is unable to...