tidychatmodels icon indicating copy to clipboard operation
tidychatmodels copied to clipboard

allow me to specify host address for ollama and maybe other vendors too

Open keneo opened this issue 1 year ago • 0 comments

I host my ollama on a separate host with strong GPU. I connect to it like this:

create_chat_ollama_custom<-function (host="127.0.0.1", port=11434) 
{
  engine <- httr2::request(base_url = glue::glue("http://{host}:{port}/api/chat"))
  chat <- list(vendor_name = "ollama", engine = engine, messages = list(), 
               params = list(stream = FALSE))
  class(chat) <- "chat"
  return(chat)
}

However I think it would be usefull also for others if you allow host parameter in your create_chat, so that we could use it like this:

create_chat('ollama', host='...my.gpu.host...')

I would implement like this: add host parameter with default value NA, and for each vendor, use vendor default address if host is NA or use specified host if provided. I.e. with dplyr::coalesce

keneo avatar Nov 18 '24 10:11 keneo