bioformats openBytes throwing exception when the preallocated buffer is bigger than enough and 3 or 4 modulo 6

openbytes can allocate a byte array (openBytes(int no, int x, int y, int w, int h)), or it can output to the beginning of a byte array (openBytes(int no, byte[] buf, int x, int y, int w, int h)). In the latter case, buf must be at least w * h * ch * bpp. However, for some values of buf.length bigger than that, I get:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 25166026 out of bounds for length 25166025
	at loci.formats.ImageTools.splitChannels(ImageTools.java:230)
	at loci.formats.ImageTools.splitChannels(ImageTools.java:198)
	at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:1053)
	at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:871)
	at loci.formats.in.MinimalTiffReader.openBytes(MinimalTiffReader.java:312)
	at loci.formats.in.TiffDelegateReader.openBytes(TiffDelegateReader.java:71)
	at loci.formats.ImageReader.openBytes(ImageReader.java:476)
	at BytesTest.main(BytesTest.java:13)

Using the file I link, if buf.length is N, where N is greater than the minimum, I either get: Index N-1 out of bounds for length N or Index N out of bounds for length N, or it does work. For the file I link, if remainder(N,6) = 3 or 4, I get one of these exceptions, otherwise it does work.

import loci.formats.ImageReader;

class BytesTest {
  public static void main(String[] args) throws Exception {
    var reader = new ImageReader();
    reader.setFlattenedResolutions(false);
    reader.setId("DeltaE_16bit_gamma2_2.tif");
    reader.setResolution(0);

    // minimum 25165824

    for (int i = 25165824; i < 33000000; i++) {
      try {
        byte[] bytes = new byte[i];
        reader.openBytes(0, bytes, 0, 0, 2048, 2048);
      } catch (Exception e) {
        System.out.println(i);
      }
    }

  }
}

which prints bad lengths.

as I said, these are 3 or 4 when divided by 6.

Could 6 be because of the number of channels times two bytes per channel value?

Test file: http://www.brucelindbloom.com/downloads/DeltaE_16bit_gamma2.2.tif.zip from http://www.brucelindbloom.com/index.html?ReferenceImages.html

Jul 24 '23 15:07 CGDogan

Hi @CGDogan, the call to open needs to be exactly the length of the plane, it is not a minimum size that will copy to the beginning of the array. The ImageTools.splitChannels for example is relying on the size of the resulting return array for its calculations.

Jul 26 '23 15:07 dgault

Hello @dgault , thank you for your response, this means that we should change the current code from verifying that the buffer is large enough to checking that the buffer is exactly of that size?

Currently openBytes implementations call checkPlaneParameters for the output buffer:

  public static void checkPlaneParameters(IFormatReader r, int no,
    int bufLength, int x, int y, int w, int h) throws FormatException
  {
    assertId(r.getCurrentFile(), true, 2);
    checkPlaneNumber(r, no);
    checkTileSize(r, x, y, w, h);
    if (bufLength >= 0) checkBufferSize(r, bufLength, w, h);
  }

calling:

  public static void checkBufferSize(IFormatReader r, int len, int w, int h)
    throws FormatException
  {
    int size = getPlaneSize(r, w, h);
    if (size > len) {
      throw new FormatException("Buffer too small (got " + len +
        ", expected " + size + ").");
    }
  }

Because it'd be safer to either make sure all code works for buffers bigger than needed or fail early with the buffer length check; currently it's likely to encounter this in production.

Jul 26 '23 15:07 CGDogan