neupy icon indicating copy to clipboard operation
neupy copied to clipboard

LevenbergMarquardt error for LSTM

Open ghost opened this issue 5 years ago • 3 comments

I'm trying to implement a simple lstm network with LevenbergMarquardt algorithm but I get an error as follow:

Cannot use 'training-updates_4/compute_jacobian/while/gradients/f_count_1' as input to 'training-updates_4/compute_jacobian/while/gradients/f_count' because they are in different while loops. See info log for more details.

here is my code:

 def create_dataset(X, y, time_steps=1):
     Xs, ys = [], []
     for i in range(len(X) - time_steps):
         v = X[i:(i + time_steps)]
         Xs.append(v)
         ys.append(y[i + time_steps])
     return np.array(Xs), np.array(ys)

 time_steps = 3
 train_x,train_y = create_dataset(inp_arr,out_arr,time_steps)

The shapes of train_x and train_y will be (87596, 3, 6) and (87596, 1) respectively. My network is as follow:

 n_time_steps = train_x[0].shape
 network = algorithms.LevenbergMarquardt(
        [ 
             layers.Input((3,6)),
             layers.LSTM(100,only_return_final=False),
             layers.LSTM(100),
             layers.Linear(1),
         ],
         verbose=True
     )
 network.train(train_x,train_y,epochs = 10)

am I doing something wrong?

ghost avatar Feb 15 '20 16:02 ghost

Hi, Can you also tell me python's version and versions of the following libraries: NeuPy, tensorflow?

itdxer avatar Feb 17 '20 14:02 itdxer

Hi, Can you also tell me python's version and versions of the following libraries: NeuPy, tensorflow?

Hi, thanks for replying to me. well, when I install the NeuPy, during the installation, it deletes the tensorflow and installs the "1.13.2" version of tensorflow and I have "0.8.2" version of the Neupy. I'm using python version 3 on google Colab.

ghost avatar Feb 17 '20 14:02 ghost

Sorry for the delay, I just tried to investigate and fix this issue, but it looks like it's not that easy (at least with tensorflow v1). Is it possible for you to use different optimizer?

Levenberg-Marquardt optimizer won't work for your dataset and architecture. It will require building N*M matrix where N is number of samples and M is number of parameters. Probably you won't be able to fit such a matrix in your computer anyway

itdxer avatar Feb 23 '20 17:02 itdxer