Local inference example... not pointing to the correct Ollama endpoint

[billyg88]

Hi, I'm trying to use Ollama for local inference but I cant seem to get the code point to the correct endpoint...

I'm using :

    assistant = VoicePipelineAgent(
        vad=ctx.proc.userdata["vad"],
        stt=deepgram.STT(),
        #llm=openai.LLM(model="gpt-4o-mini"),   !!commented out!!!
        llm = openai.LLM.with_ollama(model="llama3.1",base_url="http://xyz.io:11023/api"),
        tts=openai.TTS(),
        chat_ctx=initial_ctx,
    )

Which points the request to /api/chat/completions on my local server... I can see the request coming through ! BUT THATS NOT THE CORRECT ENDPOINT !

It should be /api/chat but I can't seem to point to that !!! It seems like the base_url is always appended with /chat/completions ??

Frustrating this one is !

Any pointers would be greatly appreciated ! This software is awesome !