Training SGPT for Custom Dataset

[rajarajanvakil]

Hi I read your paper that is cool, am trying to do this on my own dataset and my dataset is huge. Can you please tell me the exact ways to train from the scratch to achieve SGPT- both symmetric and asymmetric in both the encoder. But cross encoder would be our interest. I Have one doubt are you using bert to produce cross and BI encoder embedding. In my understanding you are using BERT as initial pipeline before fetching it to GPT to produce the cosine similarity and log probabilities please help

[Muennighoff]

Hey!

1. No BERT model is used
2. For the SGPT Cross-Encoder no training is necessary. Just use the script here. For symmetric search just change the prompt 😇