ollama-js icon indicating copy to clipboard operation
ollama-js copied to clipboard

Inconsistent parameters in `Options` interface

Open fmaclen opened this issue 1 year ago • 1 comments

Currently ollama-js has the following option types:

export interface Options {
  numa: boolean
  num_ctx: number
  num_batch: number
  num_gpu: number
  main_gpu: number
  low_vram: boolean
  f16_kv: boolean
  logits_all: boolean
  vocab_only: boolean
  use_mmap: boolean
  use_mlock: boolean
  embedding_only: boolean
  num_thread: number

  // Runtime options
  num_keep: number
  seed: number
  num_predict: number
  top_k: number
  top_p: number
  tfs_z: number
  typical_p: number
  repeat_last_n: number
  temperature: number
  repeat_penalty: number
  presence_penalty: number
  frequency_penalty: number
  mirostat: number
  mirostat_tau: number
  mirostat_eta: number
  penalize_newline: boolean
  stop: string[]
}

When compared to the parameters listed in API.md:

  • min_p is missing from Options
  • embedding_only and logits_all is present in Options but missing in API.md

This appears to also be the case with the Python library.

I'd be happy to issue PRs for both libraries if someone can confirm which parameters should be supported by each of them.

fmaclen avatar Sep 21 '24 15:09 fmaclen