Is there a way of not using ImageType constant values?

[sportbilly21]

System information

• Windows 10 19044.1826
• VS 2019, Microsoft.ML 1.71,Microsoft.ML.OnnxRuntime.GPU 12.1

I am have retrained a Tensorflow/Keras model with a resizing layer are part of the model. I converted to ONNX with opset 11. In python onnx works as expected. I am feeding with what ever size of image and models does the resizing (224, 224) and delivers the predictions.

In ML.Net I have created the pipeline and predictions engine. If I pass to the model with a 224x224 image I am getting the following error System.ArgumentException: 'Length of memory (150528) must match product of dimensions (3).' from DenseTensor.shared.cs If I pass to the model any other size of image I am getting the following System.InvalidOperationException: 'Operation is not valid due to the current state of the object.' when try to run prediction engine,

Is resizing layer not supported yes or I am doing something wrong.Please find below the code for the running the model

Source code / logs

        public void GenerateModel(string modelLocation,string inputLayer, string outputLayer, List<string> cats, int gpuID = 0)
        {
            var pipeline = this.mlContext.Transforms.ExtractPixels(inputLayer, inputColumnName: nameof(ImageData.Image), colorsToExtract: ImagePixelExtractingEstimator.ColorBits.Rgb, orderOfExtraction: ImagePixelExtractingEstimator.ColorsOrder.ARGB, interleavePixelColors: false, outputAsFloatArray: true)
              .Append(this.mlContext.Transforms.ApplyOnnxModel(outputLayer, inputLayer, modelLocation));

            var data = this.mlContext.Data.LoadFromEnumerable(new List<ImageData>());
   
            this.model = pipeline.Fit(data); //;
            this.categories = cats;
            this. Predictor = this.mlContext.Model.CreatePredictionEngine<ImageData, ImagePrediction>(this.model);

       
        }

        public void ClassifySingleImage(ImageData imgClassify)
        {

            this. Prediction = this.predictor.Predict(imgClassify);
            this. Predictions = this.prediction.Preds;

        }

Many thanks

[luisquintanilla]

Hi @sportbilly21

If you were to inspect your model using Netron what is the expected input?

Can you try adding the ResizeImages transform to your pipeline before the ExtractPixels transform.

[ghost]

This issue has been marked needs-author-action and may be missing some important information.

[sportbilly21]

Hi @luisquintanilla, I want to add the resizing layer in the model for two reasons. First to avoid the preprocessing outside the model as this gives different results depending on the processing/resizing algorithm. For example if you user ResizeImages Transform in .net but during training in python you have used opencv for resizing then you are getting different results when you are running the model in python and in .Net. Also when you add resizing layer in the model it runs of GPU saving time. I do not think the problem is the input of the model. I did some experiments. The ONNX model has issues and does not run but when I am using the Tensorflow model with the .LoadTensorFlowModel it works fine as long as the class below has a square imageType as input: public class ImageData {

    [ImageType(192, 192)]
    //public Bitmap ImgInput { get; set; }
    //[LoadColumn(0)]
    public Bitmap Image;


    //[LoadColumn(1)]
   // public string Label { get; set; }
}

If the imageType has a non square argument does not run or if it is without arguments like [ImageType()] then I am getting an error in var data = mlContext.Data.LoadFromEnumerable(new List<ImageData>()); My images I want to feed the model are squared but with different dimensions eg 190x190 ,200x200. Can I pas dynamically this size as an Argument to ImageType? inf not is there a way to avoid this ImageType column or the var data = mlContext.Data.LoadFromEnumerable(new List<ImageData>()); as pass the image to the model in a different way? Many thanks

[michaelgsharp]

Currently I dont think there is a way of doing this. This is the same as #6346. Will close this issue and track it in the other one.