euler icon indicating copy to clipboard operation
euler copied to clipboard

GraphSage的所有node的embedding都相似度非常接近

Open ZunwenYou opened this issue 6 years ago • 8 comments

1亿规模的节点,30亿左右的边,训练GraphSage模型。 训练参数如下:

--max_id 90170696
--feature_idx 0 
--feature_dim 1
--learning_rate 0.00005
--num_epochs 1
--dim 128
--batch_size 128

首先,模型的收敛性并不好,loss没怎么下降

INFO:tensorflow:loss = 437.71362, mrr = 0.39257812, step = 1488562
INFO:tensorflow:loss = 413.18103, mrr = 0.46171877, step = 1488919 (2.109 sec)
INFO:tensorflow:loss = 449.76367, mrr = 0.4251302, step = 1489304 (2.003 sec)
INFO:tensorflow:loss = 422.31223, mrr = 0.43528646, step = 1489705 (2.045 sec)
....
INFO:tensorflow:loss = 439.37802, mrr = 0.4332031, step = 1601393 (2.210 sec)
INFO:tensorflow:loss = 495.66333, mrr = 0.47460938, step = 1601788 (2.124 sec)
INFO:tensorflow:loss = 411.83685, mrr = 0.47942704, step = 1602187 (2.181 sec)
INFO:tensorflow:loss = 434.9126, mrr = 0.4126302, step = 1602586 (2.162 sec)
.....
INFO:tensorflow:loss = 432.3944, mrr = 0.45585936, step = 3520258 (2.242 sec)
INFO:tensorflow:loss = 419.18842, mrr = 0.4451823, step = 3520649 (2.169 sec)
INFO:tensorflow:loss = 439.15546, mrr = 0.3761719, step = 3521049 (2.237 sec)
INFO:tensorflow:loss = 420.41013, mrr = 0.40924478, step = 3521450 (2.228 sec)
INFO:tensorflow:loss = 425.3105, mrr = 0.4282552, step = 3521840 (2.208 sec)

其次,获得的Embedding,任意两个node的Embedding相似度超过0.96。

ZunwenYou avatar Jun 04 '19 04:06 ZunwenYou

ping @yangsiran

ZunwenYou avatar Jun 04 '19 04:06 ZunwenYou

可以调整一下学习率等参数。

yangsiran avatar Jun 04 '19 05:06 yangsiran

我们调大学习率,loss的趋势并没有改善.

ZunwenYou avatar Jun 04 '19 06:06 ZunwenYou

fanouts 设置得多少?如果层数太多,应该embedding都会接近吧

dylanwinty avatar Aug 28 '19 07:08 dylanwinty

我也遇到了类似的问题,想问一下有什么好的解决方法吗

hao920293897 avatar Sep 03 '19 02:09 hao920293897

我也遇到了类似的问题,想问一下有什么好的解决方法吗

框架没有binary特征加入运算,要把所有的特征都放在float_feature里。

ZunwenYou avatar Sep 03 '19 05:09 ZunwenYou

收敛问题最后有解决吗

oscarriddle avatar Oct 23 '20 10:10 oscarriddle

遇到了同样的问题,训练了一个异构图,模型无法收敛,且最终embbedding的余弦距离非常接近,请问如何解决呢

Ling-wei avatar Dec 09 '21 12:12 Ling-wei