ata icon indicating copy to clipboard operation
ata copied to clipboard

Add support for greater token lengths

Open danaimone opened this issue 1 month ago • 3 comments

Currently if you try to use the gpt-4o model with >2048 token length, the following exception occurs upon prompting:

thread '<unnamed>' panicked at ata/src/prompt.rs:81:20:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I am requesting that we add support for a great token length, as the pricing and performance is much better.

Reproduction:

  • Change model to gpt-4o, token length >2048
  • Try prompting

CleanShot 2024-05-14 at 09 44 03@2x

danaimone avatar May 14 '24 16:05 danaimone