Keras model model.predict for batch predictions

[nickwood0739]

Hello,

I'm trying to migrate my existing TFR code onto Keras, however I'm having difficulty getting predictions. After using

        model.fit(
            x=train_dataset,
            epochs=10,
            steps_per_epoch=self._hparams.steps_per_epoch,
            validation_steps=None,
            validation_data=vali_dataset,
            callbacks=self.build_callbacks(),
            verbose=verbose)

I then want to call model.predict on my validation dataset. However, this gives me an error from predict - "Dimensions of inputs should match: shape[0] = [32,279] vs. shape[1] = [32,291] [Op:ConcatV2] name: concat" which comes from the concatenation of predictions done by the predict method. As different ranking lists are of different length, the error is thrown. Is there a way to get predictions directly from a dataset? All the tests I've look through (eg in model) seem to call the model directly with dummy data, but the Keras documentation on predict seems to suggest this is only appropriate for small data. Is there a different method for getting batch predictions I'm missing?

Thanks, Nick

[lyyanlely]

Thank you for sharing your issue. This issue is specific to the predict function. There are two quick workarounds if you stick to your model working with full tensors:

1. You can set a list_size for the dataset so that every query in the dataset have exactly the same list_size and good to concat. For example,dataset = data.build_ranking_dataset(..., list_size=200, ...)

2. You can predict the dataset in full batch, so that no concat is needed. For example, if you know the full batch size, do dataset = dataset.batch(full_batch_size).

Otherwise, you could build and train a model running on ragged tensors, see example.