Thiago Padilha

> Shall we close #267 now that we have this? We can, this already includes all changes in #267. Only reason I did in a separate PR was to simplify...

Rebased

> If anyone is seeking for working client/server implementation, I wrote a minimal Go/Python server and client with live streaming, that is based on this awesome repo. See https://github.com/avilum/llama-saas @avilum...

> This is a large rearchitecting of the project that fundamentally changes its vision @jart @spirobel there's no rearchitecting here, in fact my goal was to introduce a server/client model...

@anzz1 I don't see how it could work using threads. There's only one instance of the model in memory, AFAIK `ggml` API is not thread-safe or supports concurrent usage (@ggerganov...

> implement this in nodejs instead of doing it in cpp @spirobel If you want to implement a server mode in another program/language such as node.js and without changes to...

> I’m personally uncomfortable with this because I don’t believe new C code should be exposed directly to the internet This TCP mode is not meant to be used directly,...

> It looks that this PR refactors current code base too much. If you consider replacing global references (stdin/stdout/stderr) with function parameters "too much refactoring", then yes. Really, review the...

Redid the commits on top of the latest C API changes. Now that the C API is implemented on llama.cpp, I've moved the program main loop to run.cpp. Seems like...

> I would like this to become a standalone example in the "./examples" folder. The main.cpp example has to remain the way it is on master. Even if you have...