cog icon indicating copy to clipboard operation
cog copied to clipboard

How to stream via webhooks?

Open schananas opened this issue 1 year ago • 1 comments

Based on documentation, if I set webook filter with "output", If I set Iterator[str] as return type and yield strings, I would expect that my webhook is called on each yield, but that is not the case, yielded output is included on the completion webhook call. As well I didnt set webhook filter "logs" but logs are still included in the end webhook call.

schananas avatar Jun 05 '24 14:06 schananas

My call:

echo '{
  "input": {},
  "webhook":"http://localhost:8080",
  "webhook_events_filter": ["start", "output" , "completed"],
  "stream": true
}' | curl -X POST -H "Content-Type: application/json" -H "Prefer: respond-async" --data-binary @- http://localhost:5000/predictions

my code:

    def predict(
        self,
  ...
    ) -> Iterator[str]:

yield 'test'

...prediction

I get 'test' in output field, but after whole prediction has finished.

schananas avatar Jun 05 '24 15:06 schananas