on-next not called when stream option is true

[yenda]

I am calling create-chat-completion with the options :stream true :on-next #(log/info :on-next %) and nothing it logs, instead I have all the "chat.completion.chunk" events dumped at once after some time

I am using azure

[yenda]

This is the code I had to use to be able to consume the stream: using aleph and manifold

(defn parse-sse [sse-string]
  (let [lines (str/split sse-string #"\n\n")]
    (for [line lines
          :let [parts (str/split line #"\n")
                data (first (filter #(str/starts-with? % "data:") parts))]]
      (when data
        (str/replace data "data: " "")))))

(defn handle-bytebuf [bytebuf]
  (let [bytes (byte-array (.readableBytes bytebuf))]
    (.getBytes bytebuf 0 bytes)
    (parse-sse (String. bytes "UTF-8"))))

(defn stream-chat-completion
  [messages options on-next]
  (let [{:keys [model temperature max-tokens]} options]
    @(d/chain
      (http/post url
                 {:headers {"api-key" key}
                  :form-params (cond-> {:temperature temperature
                                        :top_p 1
                                        :messages messages
                                        :stream true}
                                 max-tokens (assoc :max_tokens max-tokens))
                  :content-type :json
                  :pool raw-stream-connection-pool
                  })
      :body
      #(stream/consume (fn [chunk]
                         (on-next (handle-bytebuf chunk))) %))))

[wkok]

Unfortunately I don't have access to an azure account to play with.

@behrica do you experience the same issue with the current version?

[wkok]

v0.14.0 includes some changes related to SSE for openai. Could you maybe confirm if this issue still persists in azure on this new version?

[wkok]

Fixed via PR 73 in v0.21.2