chaofan
chaofan
感谢对我们工作的关注，我们会在近期开源llara的训练代码
Thank you for your suggestion, we will make modifications to this part.
Seems like you encountered errors while using multiprocessing. You could try setting `multiprocessing.set_start_method('spawn')` before `multiprocessing.Process` to address this issue.
This indeed appears to be an error caused by multiprocessing. You can `import multiprocessing` and then try setting `multiprocessing.set_start_method('spawn')`.
Could you please provide more code? We want to analyze the code to identify the cause since the specific error location is unclear.
The training pipeline is the same as the https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/reranker The dataset utilized is the same as BAAI/bge-reranker-v2-gemma and BAAI/bge-reranker-v2-minicpm-layerwise (see https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/llm_reranker) The learning rate is 4e-5, the base model is...
目前还不能使用vllm来加速bge-reranker-v2-minicpm-28的推理，您可以尝试设置参数use_bf16=True或use_fp16=True来加速推理
会去除掉句子中的停用词，同时训练的时候也会带有自回归的loss
loss是通过logits来进行计算的，通过bag of words的方式，预测句子本身/下一句所包含的tokens所在的位置的logits的概率增加
使用的是yes的logits 1. 在相同的数据下，两种方法的差异可以参考这里 https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/llm_reranker#evaluation 2. 相同数据下同一方法的不同prompt差异没有在标准的数据集上全量测试过，但是训练的时候针对不同的数据集引入了不同的prompt，所以推理的时候用prompt是有帮助的