blog icon indicating copy to clipboard operation
blog copied to clipboard

README claims of token size are outdated and inaccurate.

Open monsterswithink opened this issue 2 years ago • 1 comments

It’s important to avoid ever using claims that are dependent on thousands of variables changing each day, such as, “With a context length of over 8,000 tokens, the StarCoder models can process more input than any other open LLM,” unless you’re going to fetch this using some logic that benchmarks daily. Or include a disclaimer footnote with the date of the claim on that same line. Already OpenAI models have surpassed this three times since this was written.

monsterswithink avatar Nov 22 '23 00:11 monsterswithink

  • "OpenAI models have surpassed this three times since this was written"

  • "more input than any other Open LMM"

Km3888 avatar Apr 17 '24 16:04 Km3888