How to extract whole sentence embeddings

[GlancerZ]

Compared to the method of using the CLS token provided by BERT to extract the entire sentence embedding, is Mamba's method of placing the CLS token effective? My intuition is that the CLS token in Mamba cannot directly interact with each word's token, so its effectiveness might be poor. Therefore, would extracting the last hidden state be more effective? Thanks！