replicate-python icon indicating copy to clipboard operation
replicate-python copied to clipboard

`meta/meta-llama-3-70b` ignores `max_tokens`

Open johny-b opened this issue 1 year ago • 0 comments

I'm pretty sure I'm sending max_tokens and:

  • I get much more tokens
  • I also don't see this max_tokens when looking at my prediction in the browser

When I use exactly the same code for e.g. meta/llama-2-70b this does not happen, i.e. I really get the requested number of tokens.

johny-b avatar May 24 '24 22:05 johny-b