How to train model that takes mutiple tensors as input

[snowaterr]

I code a model for testing which takes 2 differnt tensors as input as follows:

            Tensor input_wide = keras.Input(5);
            Tensor input_deep = keras.Input(6);

            var hidden1 = keras.layers.Dense(30, activation: keras.activations.Relu).Apply(input_deep);
            var hidden2 = keras.layers.Dense(30, activation: keras.activations.Relu).Apply(hidden1);
            var concat = keras.layers.Concatenate().Apply(new Tensors(input_wide, hidden2));

            var output1 = keras.layers.Dense(1, activation: keras.activations.Relu).Apply(concat);
            var output2 = keras.layers.Dense(1, activation: keras.activations.Relu).Apply(hidden2);

            var model = keras.Model(new Tensors(input_wide, input_deep),
                new Tensors(output1, output2)
            );

            model.compile(optimizer: keras.optimizers.Adam(),
                loss: keras.losses.MeanSquaredError(),
                new[] { "accuracy" });

Here I construct a Tensors with 2 tensor, which is assigned to the inputs of model. This model can be compiled without problems. Then I try to predict by this model:

                // Just test data
                var x1 = np.array(new float[,] { { 1, 2, 3, 4, 5 }, { 1, 3, 5, 7, 9 } });
                var x2 = np.array(new float[,] { { 1, 2, 3, 4, 5, 6 }, { 1, 3, 5, 7, 9, 11 } });
                var x = new Tensors(x1, x2);
                var pred = model.predict(x);

Here I got an exception System.Collections.Generic.KeyNotFoundException:“The given key '3' was not present in the dictionary.”.Did I make a mistake or is it a bug? Many thanks.

[BalashovK]

Yes, model can be compiled but cannot be used. Unfortunately, there is only version of Predict that takes "Tensor", not "Tensors" I need to be able to feed multiple inputs into Tensorflow model too. Dear SciSharp/Tensorflow/Keras team, please consider prioritizing adding support for multi-input models. Thank you!

[ramon-garcia]

Yes, please. Not being able to train/predict models with more than one input blocks many tasks.

[AsakusaRinne]

Yes, model can be compiled but cannot be used. Unfortunately, there is only version of Predict that takes "Tensor", not "Tensors" I need to be able to feed multiple inputs into Tensorflow model too. Dear SciSharp/Tensorflow/Keras team, please consider prioritizing adding support for multi-input models. Thank you!

The issues of tf.net have been delayed for a long time because short of hands. I'd like to help on this problem. What I understand is that there is a compiled keras.Model, a list of Tensors is supposed to passed to model.fit. Is that the point of this issue?

[BalashovK]

Yes, and model.predict too My use-case is - neural network input is an image and a a float. It requires 2 separate tensors of a different shape as an input. It works in python, it works when converted to ONNX, but in it cannot be used in Tensorflow.Net. I wonder whether adding an option to pass "Tensors" instead of "Tensor" would solve the issue

[AsakusaRinne]

OK. I'll try to implement it and I'll tell you once I complete it.

[AsakusaRinne]

Now the multiple inputs of model.fit and model.predict has been supported by #996 and #1000 .

A simple example with multi-inputs LeNet could be found here.