An experimental phenomenon

[wu6shen]

trafficstars

I did an experiment that delete the propagation on graph (comment line:194-221), which means that only do dot product. It is amazing that it can get comparible results in gollaw and amazon dataset.

• Gollaw with graph: (recall@20, ndcg@20) (0.14798, 0.20794)

• Gollaw without graph: (recall@20, ndcg@20) (0.14481, 0.21080)

• Amazon with graph: (recall@20, ndcg@20) (0.03171, 0.06087)

• Amazon without graph: (recall@20, ndcg@20) (0.03295, 0.06254)

[xiangwang1223]

Sorry for the late reply after a busy week. For this finding, we think one possible reason is: (1) in the first version, we use bpr loss (implementing via sigmoid function) to do experiments and report performance in our paper. (2) in the current second version, to avoid the NAN issue, we use bpr loss (implementing via softplus function). Hence, the parameters and reports could be inconsistent with our previous and reported performance. I have changed the bpr loss back to the first version. Please try it again and let me know whether you hold the same findings. Thanks for your valuable comments.

[RileyLee95]

I did an experiment that delete the propagation on graph (comment line:194-221), which means that only do dot product. It is amazing that it can get comparible results in gollaw and amazon dataset.

• Gollaw with graph: (recall@20, ndcg@20) (0.14798, 0.20794)
• Gollaw without graph: (recall@20, ndcg@20) (0.14481, 0.21080)
• Amazon with graph: (recall@20, ndcg@20) (0.03171, 0.06087)
• Amazon without graph: (recall@20, ndcg@20) (0.03295, 0.06254)

could you please elaborate how you set up your experiment? because I found after commenting line:194-221, the initial embeddings matrix will not be updated at all.