IOError: bad message length

[chao92]

I am feeding a new undirected graph dataset with (V=18059, E=286535). It comes up with Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/queues.py", line 268, in _feed send(obj) IOError: bad message length

[tkipf]

That seems to be a python issue unrelated to this repository. Not sure if/how I can help?

On Wed 3. Oct 2018 at 16:03 Chao Jiang [email protected] wrote:

I am feeding a new undirected graph dataset with (V=18059, E=286535). It comes up with Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/queues.py", line 268, in _feed send(obj) IOError: bad message length

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tkipf/gae/issues/13, or mute the thread https://github.com/notifications/unsubscribe-auth/AHAcYL4-CL2lYRTZBlcGUEnyeH9Bs0SUks5uhNHJgaJpZM4XGOmP .

[chao92]

I am meeting another problem: Traceback (most recent call last): File "train.py", line 48, in adj_train, train_edges, val_edges, val_edges_false, test_edges, test_edges_false = mask_test_edges(adj) File "build/bdist.linux-x86_64/egg/gae/preprocessing.py", line 99, in mask_test_edges assert ~ismember(val_edges_false, edges_all) AssertionError

[yjyGo]

@chao92 I met the same problem: 'assert ~ismember(val_edges_false, edges_all) AssertionError', maybe is not uncommon. Do you resolve this matter?

[ioannisa92]

@yjyGo, @chao92 I know this is kind of late, but try and reduce the validation/test split. I did that and it worked for me. I reduced it to 5%, because my graphs are kind of small. Not sure if this is the right thing to do.

I also tried with those asserts commented out. It still ran, but, again, that is probably not recommended/right to do

[delta2323]

@yjyGo @chao92 I have encountered the same problem. I found that the cause of this error is that false validation edges can be in the test edge set. However, the above assertion requires that false validation edges be in the whole edge set, including the test edge set. I think we can solve the problem if we also check that false validation edges are not in the test edge in generating them.

Code to reproduce (test.py)

import sys
import numpy as np
import scipy
import scipy.sparse as sp
from gae import preprocessing

print(f"python: {sys.version}")
print(f"numpy: {np.__version__}")
print(f"scipy: {scipy.__version__}")
np.random.seed(4)
A = sp.csr_matrix(np.random.randint(0, 2, (10, 10)))
A = sp.triu(A, 1)
A = A + A.T
preprocessing.mask_test_edges(A)

Changes in the GAE code (to check that false validation edges is in the test edge set)

%(master)[U]git diff      
diff --git a/gae/preprocessing.py b/gae/preprocessing.py
index 9b3b093..6c1f7f8 100644
--- a/gae/preprocessing.py
+++ b/gae/preprocessing.py
@@ -95,6 +95,10 @@ def mask_test_edges(adj):
                 continue
         val_edges_false.append([idx_i, idx_j])
 
+    assert ismember([6, 0], np.array(val_edges_false))
+    assert ismember([0, 6], test_edges)
+    assert ismember([0, 6], edges_all)
+
     assert ~ismember(test_edges_false, edges_all)
     assert ~ismember(val_edges_false, edges_all)
     assert ~ismember(val_edges, train_edges)

Log

%(master)[U]python test.py
python: 3.7.5 (default, Oct 25 2019, 10:52:18) 
[Clang 4.0.1 (tags/RELEASE_401/final)]
numpy: 1.18.1
scipy: 1.3.3
Traceback (most recent call last):
  File "test.py", line 17, in <module>
    preprocessing.mask_test_edges(A)
  File "/Users/oonokenta/dev/gae/gae/preprocessing.py", line 103, in mask_test_edges
    assert ~ismember(val_edges_false, edges_all)
AssertionError