[Question] is there any documentation on how to use the worker/controller?

[bdiaz29]

Question

for instance if I wanted to call an inference from a different python script or web application?

[SuperMasterBlasterLaser]

@bdiaz29 I'm also trying to use this service in my custom script. The only hint of how to use it is shown in this testing page

You need to somehow stream your prompt in formatted way. The problem is that sending image is hard to understand. What I understood is that we need to send it as base64. But I always get this error:

Exceeds max token length. Please start a new conversation, thanks.

One user created PR #834 to add OpenAI compatible server. However it still not accepted for more than month.