Alternative to DecodeJPEG/DecodePNG

[pudg]

Is there an alternative method to create input tensors other than using DecodeJPEG or DecodePNG? I've come across this post which kinda goes along the same path i'm trying to follow, but then I run into TFException: Expects arg[0] to be uint8 but float is provided errors.

[MykolaKovalchuk]

I have loaded image, accessed its pixels, and created tensor separately from a float array, e.g.:

		TFTensor ImageToTensor(IndexedImage image)
		{
			var matrix = new float[1, image.Size.Height, image.Size.Width, 3];
			
			using (image.LockPixels(out var pixels))
			{
				for (var iy = 0; iy < image.Size.Height; iy++)
				{
					for (int ix = 0, index = iy * image.Size.Width; ix < image.Size.Width; ix++, index++)
					{
						var pixel = pixels[index];
						matrix[0, iy, ix, 0] = pixel.Blue() / 255.0f;
						matrix[0, iy, ix, 1] = pixel.Green() / 255.0f;
						matrix[0, iy, ix, 2] = pixel.Red() / 255.0f;
					}
				}
			}

			TFTensor tensor = matrix;
			return tensor;
		}

Here IndexedImage is my own class, and to load images I use System.Graphics. Before this I did my own image cropping and scaling as needed. And for training in Python I used OpenCV to load and transform images, so colors order here is BGR.

[sdg002]

Looks good. Thanks. Any reason why you did not use System.Drawing.Bitmap

[MykolaKovalchuk]

System.Drawing.Bitmap stores pixels in unmanaged memory, and accessing them is cumbersome and slow. My class IndexedImage just wraps an array of pixels with quick and simple access, and I have some simple image transformation methods written and optimised to use this class. And I use System.Drawing.Bitmap only to load/save image to file. You can check it here and here.

[sdg002]

Good stuff. Thank you.

[sdg002]

Sorry. One more basic question. I can see how you are using the IndexedClass (e.g. ImageCopier). Could you please point me to the file where you are populating/constructing an instance of IndexedImage? I would imagine the source information for IndexedImage would be a Bitmap class.

Many thanks.

[MykolaKovalchuk]

Bitmap to/from IndexedImage code is in IndexedImageExtensions.cs (updated version with load/save to files here)

[sdg002]

Cool. Thank you.

[dhanarfmlx]

I will just leave it here: convert bitmap to tensor

Advantage: Self contained, no need additional dependency Disadvantage: Uses unsafe for performance reason

private TFTensor ConvertToTensor( Bitmap image )
{
  var depth = Image.GetPixelFormatSize( image.PixelFormat ) / 8;

  if( depth != 3 )
  {
	throw new NotSupportedException( "Only 24-bit images are supported" );
  }

  var matrix = new byte[1, image.Height, image.Width, 3];

  var data = image.LockBits( new Rectangle( 0, 0, image.Width, image.Height ), ImageLockMode.ReadOnly, image.PixelFormat );

  try
  {
	unsafe
	{
	  Parallel.For( 0, image.Height, y =>
	  {
		var line = (byte*)data.Scan0 + y * data.Stride;

		for( var x = 0; x < data.Width; x++ )
		{
		  var red = line[x * depth];
		  var green = line[x * depth + 1];
		  var blue = line[x * depth + 2];

		  matrix[0, y, x, 0] = blue;
		  matrix[0, y, x, 1] = green;
		  matrix[0, y, x, 2] = red;
		}
	  } );
	}
  }
  finally
  {
	image.UnlockBits( data );
  }

  return matrix;
}