aici icon indicating copy to clipboard operation
aici copied to clipboard

better error on prompt len exceeded in rllm

Open mmoskal opened this issue 1 year ago • 0 comments

The following gives stop_reason: "failed" but no indication why.

import pyaici.server as aici

async def main():
    await aici.FixedTokens("Some long text that will serve as a prompt for the model to generate more text. This is a test of the AICI server." * 500)
    await aici.gen_text(max_tokens=250, store_var=f"function", stop_at="```")

aici.start(main())

mmoskal avatar Mar 25 '24 17:03 mmoskal