LongChat
LongChat copied to clipboard
About the print message
Hi, the printed message I can not understand.
I set
def __init__(
self, dim, ratio, max_position_embeddings=2048, base=10000, device=None
):
ratio=2, max_position_embeddings=1024
Since my GPU can not fit minimal 2048, so, I thought it was expanding the context from 1024 to 2048
But I got print like this:
Condensing Positional embeddings from 4096 to 2048
Which I don't understand?