How to cancel closure openAI.chatsStream(query: query)

[iTarek]

Is there any way to cancel chatsStream closure?

openAI.chatsStream(query: query) { partialResult in
    switch partialResult {
    case .success(let result):
        print(result.choices)
    case .failure(let error):
        //Handle chunk error here
    }
} completion: { error in
    //Handle streaming error here
}

I call this function inside SwiftUI OnAppear and when the sheet dismissed the closure keep running, is there any solution for that?

[ljoukov]

Yes, nothing special is needed, just cancel the Task which runs the query.

For myself I got a little wrapper as I feel I needed a bit more obvious semantics:

@MainActor
class ChatQueryTask: ObservableObject {
  private var task: Task<Void, Never>?

  enum Status {
    case completed
    case interrupted
    case failed
  }

  func query(
    messages: [Chat],
    onContent: @MainActor @escaping (String) -> Void,
    onFinished: @MainActor @escaping (Status) -> Void) async
  {
    if let task = task {
      let _ = await task.result
    }
    let newTask = Task {
      let query = ChatQuery(model: .gpt4, messages: messages)
      do {
        var text = ""
        for try await result in openAI.chatsStream(query: query) {
          if let choice = result.choices.first {
            if let content = choice.delta.content {
              text += content
              onContent(text)
            }
          }
        }
        let isCancelled = Task.isCancelled
        onFinished(isCancelled ? .interrupted : .completed)
      } catch {
        onFinished(.failed)
      }
    }
    task = newTask
    let _ = await newTask.value
    task = nil
  }

  func cancel() async {
    if let task = task {
      task.cancel()
      let _ = await task.result
    }
  }
}

[cenkalti]

If my understanding about Swift is correct, above solution will not work. This is because tasks are cooperative, meaning they are not preemptively interrupted or canceled by the system. Instead, a task must check for cancellation points or explicitly support cancellation for it to be interrupted from outside. You can request cancellation on a task from outside, but it is up to the task itself to periodically check if a cancellation has been requested and gracefully terminate if so. I don't see any cancellation is being handled inside chatsStream method. The method uses URLSession inside but I don't see it any calls to URLSession.cancel().