ollama-js icon indicating copy to clipboard operation
ollama-js copied to clipboard

Is there any way to stop the response using this lib?

Open ukicomputers opened this issue 6 months ago • 1 comments

Hi! Thanks for great library! Is there any way to stop the response using this lib? We are low on resources, so we want when user clicks stop button on our frontend (or closes the page) to stop generating response. We've tried this, but it didn't end the response flow:

try {
    for await (const part of response) {
      if (clientDisconnected) {
        break; // Stop sending data if the client disconnects
      }
      res.write(`${JSON.stringify(part)}`);
    }
  } catch (err) {
    console.error('Error while streaming response:', err);
  } finally {
    console.log("Response ended");
    res.end();
  }

Thanks in advance!

ukicomputers avatar Aug 09 '24 08:08 ukicomputers