FlagEmbedding icon indicating copy to clipboard operation
FlagEmbedding copied to clipboard

Hard negatives运行时一直卡顿,没有正常输出

Open karong398 opened this issue 1 year ago • 2 comments

运行命令 python -m FlagEmbedding.baai_general_embedding.finetune.hn_mine
--model_name_or_path '/Volumes/移动硬盘/ptrain/output/encoder_model'
--input_file toy_finetune_data.jsonl
--output_file toy_finetune_data_minedHN.jsonl
--range_for_sampling 1-200
--negative_number 15

非GPU运行时,一直卡顿在 _torch_pytree._register_pytree_node( inferencing embedding for corpus (number=15)-------------- inferencing embedding for queries (number=10)-------------- create index and search------------------

karong398 avatar May 05 '24 13:05 karong398

[ 卡顿 ](ur

karong398 avatar May 05 '24 13:05 karong398

@karong398 , if you have many candidates, searching using the CPU takes a lot of time. You can use GPUs or reduce the size of the corpus to speed up the searching process.

staoxiao avatar May 06 '24 02:05 staoxiao