neural-cherche icon indicating copy to clipboard operation
neural-cherche copied to clipboard

Published 20 hours ago • verified publisher icon raphaelsty

[Feature request] ColBERT V2

Open filippo82 opened this issue 1 year ago • 3 comments

Hi @raphaelsty,

first of all, thanks a lot for your this project. I really appreciate its simplicity and effectiveness.

Question: do you have any plans to implement ColBERT V2?

Best wishes.

filippo82 avatar Feb 17 '24 21:02 filippo82

Hi @filippo82, I think it could be cool to add distillation loss of course. I plan to improve the loss function of the train module in the following weeks, there won't be any breaking change.

Before releasing the centroids algorithm of ColBERTV2 I plan to release another method to accelerate ColBERT retriever, it will be really fast and still accurate, work in progress.

raphaelsty avatar Feb 18 '24 17:02 raphaelsty

Hi @raphaelsty 👋🏻 thanks a lot for your reply and sorry for the slooow reply.

Let me know if there is any way I can help with testing/debugging.

filippo82 avatar Mar 05 '24 12:03 filippo82

Hi @filippo82, I did release neural-cherche 1.1.0 which improve loss stability and brings better default parameters to models. Also I did release neural-tree in order to accelerate ColBERT.

Feel free to open a PR in neural-cherche if you are interested in Knowledge Distillation

raphaelsty avatar Mar 05 '24 17:03 raphaelsty