keras-gcn icon indicating copy to clipboard operation
keras-gcn copied to clipboard

ValueError: setting an array element with a sequence.

Open manohareddy opened this issue 5 years ago • 8 comments

Loading cora dataset... Dataset has 2708 nodes, 5429 edges, 1433 features. Using local pooling filters... WARNING:tensorflow:From /Users/manohar/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.


ValueError Traceback (most recent call last) in () 66 epochs=1, 67 shuffle=False, ---> 68 verbose=0) 69 70 # Predict on full dataset

~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs) 1237 steps_per_epoch=steps_per_epoch, 1238 validation_steps=validation_steps, -> 1239 validation_freq=validation_freq) 1240 1241 def evaluate(self,

~/anaconda3/lib/python3.6/site-packages/keras/engine/training_arrays.py in fit_loop(model, fit_function, fit_inputs, out_labels, batch_size, epochs, verbose, callbacks, val_function, val_inputs, shuffle, initial_epoch, steps_per_epoch, validation_steps, validation_freq) 194 ins_batch[i] = ins_batch[i].toarray() 195 --> 196 outs = fit_function(ins_batch) 197 outs = to_list(outs) 198 for l, o in zip(out_labels, outs):

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/keras/backend.py in call(self, inputs) 3275 tensor_type = dtypes_module.as_dtype(tensor.dtype) 3276 array_vals.append(np.asarray(value, -> 3277 dtype=tensor_type.as_numpy_dtype)) 3278 3279 if self.feed_dict:

~/anaconda3/lib/python3.6/site-packages/numpy/core/_asarray.py in asarray(a, dtype, order) 83 84 """ ---> 85 return array(a, dtype, copy=False, order=order) 86 87

ValueError: setting an array element with a sequence.

I am trying to run the code as is but getting this error. The code ran fine 2 weeks back, but I am having trouble now.

Do you think its tensorflow or keras versions?

manohareddy avatar Oct 28 '19 22:10 manohareddy

I ran into the same issue. I think its an issue with sparse matrix input. https://stackoverflow.com/questions/46579164/use-sparse-input-in-keras-with-tensorflow

sunnyqywang avatar Nov 25 '19 17:11 sunnyqywang

Have you solve this problem? When I run "train.py", I got this problem (ValueError: setting an array element with a sequence.)


Epoch 1/200 Traceback (most recent call last): File "E:/GCN_Keras-master/train.py", line 85, in callbacks=[es_callback]) File "D:\Anaconda\software\lib\site-packages\keras\engine\training.py", line 1239, in fit validation_freq=validation_freq) File "D:\Anaconda\software\lib\site-packages\keras\engine\training_arrays.py", line 196, in fit_loop outs = fit_function(ins_batch) File "D:\Anaconda\software\lib\site-packages\tensorflow\python\keras\backend.py", line 3277, in call dtype=tensor_type.as_numpy_dtype)) File "D:\Anaconda\software\lib\site-packages\numpy\core\numeric.py", line 538, in asarray return array(a, dtype, copy=False, order=order) ValueError: setting an array element with a sequence.


Is this problem really caused by the inputs? I mean the inout array is not aligned?

LancelotQAQ avatar Apr 08 '20 01:04 LancelotQAQ

It seems to be an issue with the sparse=True in the Input layer. There are 2 possible solutions:

  1. Remove the sparse=True argument in the Input layer (var named G).
  2. convert your adjacency matrix from scipy.sparse csr_matrix to a tf.SparseTensor (step1). You're going to have to change the model.fit (step2) and model.predict (step3) a little bit. All steps below:

step1

def convert_sparse_matrix_to_sparse_tensor(X):
    # got from https://stackoverflow.com/questions/40896157/scipy-sparse-csr-matrix-to-tensorflow-sparsetensor-mini-batch-gradient-descent
    coo = X.tocoo()
    indices = np.mat([coo.row, coo.col]).transpose()
    return tf.SparseTensor(indices, coo.data, coo.shape)

# apply this before training
graph[1] = convert_sparse_matrix_to_sparse_tensor(graph[1])

step2

# adjust model.fit to use tf.Tensor (sparse in this case) 
model.fit(graph, y_train, sample_weight=train_mask,
              steps_per_epoch=1, epochs=1, shuffle=False, verbose=0)

step3

# adjust model.predict also
preds = model.predict(graph, steps=1)

I'm not sure these solutions don't affect the end result as I couldn't run the original code.

Falcatrua avatar Apr 30 '20 18:04 Falcatrua

It seems to be an issue with the sparse=True in the Input layer. There are 2 possible solutions:

  1. Remove the sparse=True argument in the Input layer (var named G).
  2. convert your adjacency matrix from scipy.sparse csr_matrix to a tf.SparseTensor (step1). You're going to have to change the model.fit (step2) and model.predict (step3) a little bit. All steps below:

step1

def convert_sparse_matrix_to_sparse_tensor(X):
    # got from https://stackoverflow.com/questions/40896157/scipy-sparse-csr-matrix-to-tensorflow-sparsetensor-mini-batch-gradient-descent
    coo = X.tocoo()
    indices = np.mat([coo.row, coo.col]).transpose()
    return tf.SparseTensor(indices, coo.data, coo.shape)

# apply this before training
graph[1] = convert_sparse_matrix_to_sparse_tensor(graph[1])

step2

# adjust model.fit to use tf.Tensor (sparse in this case) 
model.fit(graph, y_train, sample_weight=train_mask,
              steps_per_epoch=1, epochs=1, shuffle=False, verbose=0)

step3

# adjust model.predict also
preds = model.predict(graph, steps=1)

I'm not sure these solutions don't affect the end result as I couldn't run the original code.

your second met the error, and the stack is as follows,

Traceback (most recent call last):
  File "D:\JetBrains\Toolbox\apps\PyCharm-P\ch-0\201.6668.115\plugins\python\helpers\pydev\pydevd.py", line 1438, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "D:\JetBrains\Toolbox\apps\PyCharm-P\ch-0\201.6668.115\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "E:/1-Research/0-DP+GCN/0/5GNNs/keras-gcn/kegra/train.py", line 88, in <module>
    steps_per_epoch=1, epochs=1, shuffle=False, verbose=0)
  File "D:\Anaconda\envs\tf14\lib\site-packages\keras\engine\training.py", line 1239, in fit
    validation_freq=validation_freq)
  File "D:\Anaconda\envs\tf14\lib\site-packages\keras\engine\training_arrays.py", line 152, in fit_loop
    outs = fit_function(fit_inputs)
  File "D:\Anaconda\envs\tf14\lib\site-packages\tensorflow\python\keras\backend.py", line 3262, in __call__
    sparse_coo = value.tocoo()
AttributeError: 'SparseTensor' object has no attribute 'tocoo'

Billy1900 avatar May 22 '20 12:05 Billy1900

It looks like you have already converted to tf.SparseTensor before passing to the convert function. I'm sharing a gist with the whole training code: https://gist.github.com/Falcatrua/fc4ed4d2cb33f08acf54bdf12c45d641 Check if it works for you

Falcatrua avatar May 22 '20 13:05 Falcatrua

It looks like you have already converted to tf.SparseTensor before passing to the convert function. I'm sharing a gist with the whole training code: https://gist.github.com/Falcatrua/fc4ed4d2cb33f08acf54bdf12c45d641 Check if it works for you

For the first solution, the accuracy is too low if you remove sparse=True

Additionally, the second one failed.

Stack:

Traceback (most recent call last):
  File "train.py", line 91, in <module>
    steps_per_epoch=1, epochs=1, shuffle=False, verbose=0)
  File "D:\Anaconda\envs\tf14\lib\site-packages\keras\engine\training.py", line 1239, in fit
    validation_freq=validation_freq)
  File "D:\Anaconda\envs\tf14\lib\site-packages\keras\engine\training_arrays.py", line 152, in fit_loop
    outs = fit_function(fit_inputs)
  File "D:\Anaconda\envs\tf14\lib\site-packages\tensorflow\python\keras\backend.py", line 3262, in __call__
    sparse_coo = value.tocoo()
AttributeError: 'SparseTensor' object has no attribute 'tocoo'

Billy1900 avatar May 23 '20 06:05 Billy1900

It looks like you have already converted to tf.SparseTensor before passing to the convert function. I'm sharing a gist with the whole training code: https://gist.github.com/Falcatrua/fc4ed4d2cb33f08acf54bdf12c45d641 Check if it works for you

For the first solution, the accuracy is too low if you remove sparse=True

Additionally, the second one failed.

Stack:

Traceback (most recent call last):
  File "train.py", line 91, in <module>
    steps_per_epoch=1, epochs=1, shuffle=False, verbose=0)
  File "D:\Anaconda\envs\tf14\lib\site-packages\keras\engine\training.py", line 1239, in fit
    validation_freq=validation_freq)
  File "D:\Anaconda\envs\tf14\lib\site-packages\keras\engine\training_arrays.py", line 152, in fit_loop
    outs = fit_function(fit_inputs)
  File "D:\Anaconda\envs\tf14\lib\site-packages\tensorflow\python\keras\backend.py", line 3262, in __call__
    sparse_coo = value.tocoo()
AttributeError: 'SparseTensor' object has no attribute 'tocoo'

Facing the same issue

liaq192 avatar Jun 04 '20 15:06 liaq192

hello for me too i have the same problem , if any one solved it.

saleems11 avatar Jun 13 '20 11:06 saleems11