Is there any way to stop the response using this lib?

[ukicomputers]

Hi! Thanks for great library! Is there any way to stop the response using this lib? We are low on resources, so we want when user clicks stop button on our frontend (or closes the page) to stop generating response. We've tried this, but it didn't end the response flow:

try {
    for await (const part of response) {
      if (clientDisconnected) {
        break; // Stop sending data if the client disconnects
      }
      res.write(`${JSON.stringify(part)}`);
    }
  } catch (err) {
    console.error('Error while streaming response:', err);
  } finally {
    console.log("Response ended");
    res.end();
  }

Thanks in advance!

[hopperelec]

Yes, you can either abort all ongoing request (see any request example) or you can abort a specific request (see specific request example)

[gabrielBusuioc]

Hi! Is it also possible to abort a non-streamable chat/generate request?

Will the request be actually aborted using the code snippet below?

const olm = new Ollama();

setTimeout(() => {
  olm.abort()
}, 100) // asume that in 100 ms the user will trigger a new request =>the prior request is not of interest anymore

olm.chat({ // or olm.generate
    model: 'llama3.2',
    prompt: 'Write me a long long story',
    stream: false,
  }).then(
    async (resp) => {
      // do smth with the response
    }
  )
)

Thanks in advance!