phartman-keysight

1. So you don't recommend adding instructions "Represent X for finding Y" when embedding documents or queries? 2. If not, do you recommend adding instructions like that if we finetune?

Okay so use instructions for document retrieval, just for the query embedding side not the document embedding side. Thanks for the excerpt I understand the one sided instructions now. Do...

I know standard grit generally uses mean pooling of the last hidden state for embeddings. I know it can use weighted mean, CLS, or last token instead. I know mean...

Yeah, I see what you're saying. The reason the language model head works is because it only uses the last token embedding to generate the logits for the next token....