Error while using Custom client

[mallowhjs]

TypeError: itr.getReader is not a function node_modules/ollama/dist/utils.js:120:28)

My code : const ollama = new Ollama({ host: 'https://...' }) const ollamaResponse = await ollama.chat(apiMessage);

Called from NodeJs, but it looks like the code is using browser libraries. Do I miss something ? Any idea to solve that issue ?

[Spankyzone]

Try this code:

import ollama from 'ollama';
const message = {
  role: 'user',
  content: `Tell me a joke`,
};

const response = await ollama.chat({
  model: 'mistral:latest',
  messages: [message],
  stream: true,
});
for await (const part of response) {
  process.stdout.write(part.message.content);
}

I use this in nodeJS, it works. Your package.json file must contain this also "type": "module"

[bouchtaoui-dev]

I've installed Ollama on my little server and I'm running Mistral. Everything works fine, I can chat via cmdline interface, and also API requests works ok, except streaming response. I try to get streaming response working by using your library, but I got stuck using it.

I have this simple script to talk to my local Ollama instance:

import {Ollama} from 'ollama'

async function run() {
    const ollama = new Ollama({host: 'http://xxxx.duckdns.org:11434'})
    
    const response = await ollama.generate({
        model: 'mistral',
        prompt: "Write a C code that parses a string into words that are divided by spaces.",
        stream: true
    });
    
    console.log(response);

    for await(const part of response) {
        process.stdout.write(part.message.content);
    }
}

run();

I get the following error:

{
  next: [Function (anonymous)],
  throw: [Function (anonymous)],
  return: [Function (anonymous)],
  [Symbol(Symbol.asyncIterator)]: [Function (anonymous)]
}
file:///Users/nordin/development/nodejs/ollama-req/main.js:15
        process.stdout.write(part.message.content);
                                          ^

TypeError: Cannot read properties of undefined (reading 'content')
    at run (file:///Users/nordin/development/nodejs/ollama-req/main.js:15:43)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

I hope you can pinpoint the problem, as it's relatively simple piece of script.

[Spankyzone]

I have the same problem when I used ollama in the browser. I think you need to call next() func in returned object. Try this:

const responseNext = await response.next()
for await(const part of responseNext) {
    process.stdout.write(part.message.content);
}

[bouchtaoui-dev]

You helped me to take a look at the iterator I wasn't familiar with. The following snippet should give you a working result:

    for await (const part of response) {
        console.log(part);
    }

From here on you're good to go 😉

[chergn]

太优秀了