t2v-transformers-models
t2v-transformers-models copied to clipboard
The tokenizer max length should be equal to model max length
I don't know any model transformer models where the where the model max length is 500. Second, I know that any model with a max length should be equal to power of 2 if it's on a gpu.
https://github.com/weaviate/t2v-transformers-models/blob/7709cdef264d240af71a1c83e8dc1aec540a376e/vectorizer.py#L64C1-L65C64