aitextgen
aitextgen copied to clipboard
Published
20 hours ago •
minimaxir
minimaxir
Get embeddings from model
Two distinct use cases:
- Get embeddings corresponding to a specific token (e.g. the embeddings corresponding to an
<|endoftext|>token.) - Get embeddings for a full input text word2vec style.
Technically somewhat outside the scope for aitextgen but it's not in base Transformers and I can make clever use of it.
