gemma-2B-10M ```generate()``` in main.py seems only processes the last 2048 tokens of the input prompt ?

```generate()``` in main.py seems only processes the last 2048 tokens of the input prompt ?

Open MrYxJ opened this issue 1 month ago • 0 comments

generate() in main.py seems only processes the last 2048 tokens of the input prompt ?

https://github.com/mustafaaljadery/gemma-2B-10M/blob/cb97c2f686a41d4d54c259437dcdcd4f7f8da5f0/src/main.py#L15C9-L15C54

If prompt is entered with a length greater than 2048, then writing generate seems to truncate with only the last 2048 tokens, which seems wrong? Did I misunderstand?

May 26 '24 10:05 MrYxJ

gemma-2B-10M gemma-2B-10M copied to clipboard

```generate()``` in main.py seems only processes the last 2048 tokens of the input prompt ?

gemma-2B-10M
gemma-2B-10M copied to clipboard