GDL_code
GDL_code copied to clipboard
Mismatch size rasied at categorical_crossentropy
At chap7 Attention RNN training period,here i follow the book to build the model looks like this:
notes_in = Input(shape=(None,))
durations_in = Input(shape=(None,))
embedded_size = 100
rnn_units = 256
x1 = Embedding(n_notes,embedded_size)(notes_in)
x2 = Embedding(n_durations,embedded_size)(durations_in)
x = Concatenate(axis=2)([x1,x2])
x = LSTM(rnn_units,return_sequences=True)(x)
x = LSTM(rnn_units,return_sequences=True)(x)
e = Dense(1,activation='tanh')(x)
e = Reshape([-1])(e)
alpha = Activation('softmax')(e)
c = Permute([2,1])(x)
c = Multiply()([x,c])
c = Lambda(lambda xin: K.sum(xin, axis=1), output_shape=(rnn_units,))(c)
notes_output = Dense(n_notes,activation='softmax',name='pitch')(c)
durations_output = Dense(n_durations,activation='softmax',name='duration')(c)
att_model = Model([notes_in, durations_in], alpha)
optimizer = RMSprop(lr=0.001)
# build model
att_model.compile(loss=['categorical_crossentropy','categorical_crossentropy'],optimizer=optimizer)
But when i use fit function to train it,it raise an mismatch tensor size on categorical_crossentropy calculation,like below:
ValueError: in user code:
File "/root/anaconda3/envs/tf2/lib/python3.9/site-packages/keras/src/engine/training.py", line 1338, in train_function *
...
File "/root/anaconda3/envs/tf2/lib/python3.9/site-packages/keras/src/backend.py", line 5560, in categorical_crossentropy
target.shape.assert_is_compatible_with(output.shape)
ValueError: Shapes (None, 470) and (None, 32) are incompatible
I had finish to train the double LSTM model without attention strategy,so here i want to train it with attention,it raise this,how should i correct the code,everything were follow on the book and code repo.