EmbedKGQA icon indicating copy to clipboard operation
EmbedKGQA copied to clipboard

pre-train accuracy for fbwq_half

Open lihuiliullh opened this issue 3 years ago • 6 comments

May I know how much the accuracy is when using kge trains fbwq_half ? Also, when you use kgc to train fbwq_half, did you only use train.txt?

lihuiliullh avatar Jan 09 '22 02:01 lihuiliullh

I used the pre-trained embedding on KGQA, the accuracy is very low. How did you learn the embedding use kge? @apoorvumang

lihuiliullh avatar Jan 10 '22 16:01 lihuiliullh

Can you please elaborate on what experiment you did, what commands you used?

apoorvumang avatar Jan 10 '22 16:01 apoorvumang

I try to use kge to get the pre-train embedding of fbwq_half, and use the pre-trained embedding in multi-hop answering on fbwq_half. @apoorvumang

lihuiliullh avatar Jan 10 '22 17:01 lihuiliullh

But the pre-train accuracy is very low when using kge. If I use that embedding, the kgqa accuracy is also much lower than the result in the paper. So, I want to know, how do you use kge to pre-train the model. The config file I use is listed below.

complex: entity_embedder: dropout: 0.44299429655075073 regularize_weight: 7.830760727899156e-12 relation_embedder: dropout: -0.4746062345802784 regularize_weight: 1.182876478423781e-10 dataset: name: fbwq_half eval: batch_size: 200 chunk_size: 25000 num_workers: 2 import:

  • complex lookup_embedder: dim: 400 initialize_args: normal_: mean: 0.0 std: 5.8970567449527816e-05 regularize_args: p: 1 weighted: true sparse: true model: complex negative_sampling: implementation: batch num_samples: o: 7851 s: 2176 shared: true with_replacement: false train: auto_correct: true batch_size: 1024 loss_arg: .nan lr_scheduler: ReduceLROnPlateau lr_scheduler_args: factor: 0.95 mode: max patience: 1 threshold: 0.0001 max_epochs: 200 num_workers: 8 optimizer_args: lr: 0.6560544891789137 type: negative_sampling valid: early_stopping: min_threshold: epochs: 10 metric_value: 0.1 patience: 10

lihuiliullh avatar Jan 10 '22 17:01 lihuiliullh

Thanks for the details, let me try it with this config and I'll get back to you

apoorvumang avatar Jan 11 '22 05:01 apoorvumang

Can you share your config file to me? so I can get the same performance in the paper @apoorvumang

lihuiliullh avatar Jan 11 '22 06:01 lihuiliullh