Calculating "fit over" dimensions (fit on either axis)

[mindplay-dk]

Feature request

What are you trying to achieve?

Calculate "fit over" dimensions.

This is the term that I recall some image programs using in the past - I can't really explain why it's named that way, but basically this means:

Constrain the width and height to a given "size" on either axis.

So, if the image is wide, constrain the width - if it's tall, constrain the height.

This is useful when you want to constrain the overall number of pixels - for example, assuming you have mixed content with both portrait and landscape images, let's say you want to produce full-screen images to use on phones and tablets, where the display can be rotated.

When you searched for similar feature requests, what did you find that might be related?

Nuthin'.

What would you expect the API to look like?

I don't know if this is a good fit for the existing resize options API - having another option could get confusing, as this likely won't "play nice" or make sense in combination with some of the other existing options.

What alternatives have you considered?

The better solution might be another documentation entry.

Here's what I came up with:

function fitTo(size, width, height, { withoutEnlargement, withoutReduction } = {}) {
  let ratio = width > height
    ? size / width
    : size / height;

  if (withoutEnlargement) {
    ratio = Math.min(ratio, 1);
  }

  if (withoutReduction) {
    ratio = Math.max(ratio, 1);
  }
  
  return {
    width: Math.round(width * ratio),
    height: Math.round(height * ratio),
    ratio
  }
}

This will constrain the given width or height to a given size, while preserving proportions.

I added options to constrain the resulting width and height to ratios either withoutEnlargement or withoutReduction - these are identical to how the resize options work.

I'm not certain if these options are necessary - I mean, you could just pass the unconstrained dimensions and the same options to resize after, so maybe this is enough:

function fitTo(size, width, height) {
  let ratio = width > height
    ? size / width
    : size / height;
  
  return {
    width: Math.round(width * ratio),
    height: Math.round(height * ratio),
    ratio
  }
}

Alternatively, maybe we could add a size option to resize, although as said, this might get confusing, since it would have to ignore width and height if size is specified.

This function requires you first obtain the width and height from metadata, which could be an argument for actually including this feature in the API somehow - if we just add an example to the documentation, it's hard to say if it belongs in documentation for resize or metadata. (If you're trying to resize an image to fit, you're most likely looking at the documentation for resize - but the function itself requires information from metadata, so which does it relate more to?)

[lovell]

This sounds like resizing to fit "inside" the provided dimensions.

https://sharp.pixelplumbing.com/api-resize#resize

inside Preserving aspect ratio, resize the image to be as large as possible while ensuring its dimensions are less than or equal to both those specified.

.resize({ width: size, height: size, fit: 'inside' }

[mindplay-dk]

This does in fact work - but the thing is, I do need to calculate the actual size, because I'm also using the ratio to decide whether or not to serve a resized version of the image; if the calculated ratio is over 0.8, for example, I don't want to create a resized version almost the same size, and instead I'll redirect to the original full-size image.

Since resize can internally calculate the dimensions, perhaps the most intuitive thing would be an option to only calculate sizes, but not actually resize - using the same arguments?

Something like .getSize({ width: size, height: size, fit: 'inside' }, which would internally run the same calculations using metadata but just return the result without actually resizing?

[ssendev]

Since resize can internally calculate the dimensions, perhaps the most intuitive thing would be an option to only calculate sizes, but not actually resize - using the same arguments?

Yes please! I tried to manually write that function but no matter whether i use round floor or ceil there always seem to be cases where the rounding is different so the actual image is 1 pixel off. Which is quite noticeable with <img src="i.jpg" width=50 height=20> if the actual image is 21 pixel high and is text.

Something like .getSize({ width: size, height: size, fit: 'inside' }, which would internally run the same calculations using metadata but just return the result without actually resizing?

Not quite since this should be fast and without requiring an actual image so a freestanding function which allows to give the original dimensions would be better.

[ssendev]

Ok this was definitely more involved than I anticipated but here it is:

function resizeSize(
  width: number,
  height: number,
  targetWidth: number | undefined = 0,
  targetHeight: number | undefined = 0,
  fit: 'cover' | 'contain' | 'fill' | 'inside' | 'outside',
  withoutEnlargement: boolean,
  withoutReduction: boolean,
  format: 'jpg' | 'svg' | 'pdf' | 'webp' | string,
  fastShrinkOnLoad: boolean,
  noShrinkOnLoad?: boolean
): [number, number] {
  let [hshrink, vshrink] = resolveShrink(
    width,
    height,
    targetWidth,
    targetHeight,
    fit,
    withoutEnlargement,
    withoutReduction
  )

  // The jpeg preload shrink.
  let jpegShrinkOnLoad = 1

  // WebP, PDF, SVG scale
  let scale = 1.0

  // Try to reload input using shrink-on-load for JPEG, WebP, SVG and PDF, when:
  //  - the width or height parameters are specified;
  //  - gamma correction doesn't need to be applied;
  //  - trimming or pre-resize extract isn't required;
  //  - input colourspace is not specified;
  const shouldPreShrink =
    (targetWidth > 0 || targetHeight > 0) && !noShrinkOnLoad
  // && baton->gamma == 0 && baton->topOffsetPre == -1 && baton->trimThreshold == 0.0 &&
  //  baton->colourspaceInput == VIPS_INTERPRETATION_LAST && !shouldRotateBefore;

  if (shouldPreShrink) {
    // The common part of the shrink: the bit by which both axes must be shrunk
    const shrink = Math.min(hshrink, vshrink)

    if (format === 'jpg') {
      // Leave at least a factor of two for the final resize step, when fastShrinkOnLoad: false
      // for more consistent results and to avoid extra sharpness to the image
      const factor = fastShrinkOnLoad ? 1 : 2
      if (shrink >= 8 * factor) {
        jpegShrinkOnLoad = 8
      } else if (shrink >= 4 * factor) {
        jpegShrinkOnLoad = 4
      } else if (shrink >= 2 * factor) {
        jpegShrinkOnLoad = 2
      }
      // Lower shrink-on-load for known libjpeg rounding errors
      if (jpegShrinkOnLoad > 1 && Math.round(shrink) == jpegShrinkOnLoad) {
        jpegShrinkOnLoad /= 2
      }
    } else if (format === 'webp' && shrink > 1.0) {
      // Avoid upscaling via webp
      scale = 1.0 / shrink
    } else if (format === 'svg' || format === 'pdf') {
      scale = 1.0 / shrink
    }
  }

  let inputWidth
  let inputHeight

  if (scale !== 1.0 || jpegShrinkOnLoad > 1) {
    // Size after pre shrinking
    if (jpegShrinkOnLoad > 1) {
      inputWidth = Math.floor(width / jpegShrinkOnLoad)
      inputHeight = Math.floor(height / jpegShrinkOnLoad)
    } else {
      inputWidth = Math.round(width * scale)
      inputHeight = Math.round(height * scale)
    }

    const shrunk = resolveShrink(
      inputWidth,
      inputHeight,
      targetWidth,
      targetHeight,
      fit,
      withoutEnlargement,
      withoutReduction
    )
    hshrink = shrunk[0]
    vshrink = shrunk[1]

    // Size after shrinking
    inputWidth = Math.round(inputWidth / hshrink)
    inputHeight = Math.round(inputHeight / vshrink)
  } else {
    // Size after shrinking
    inputWidth = Math.round(width / hshrink)
    inputHeight = Math.round(height / vshrink)
  }

  // Resolve dimensions
  if (!targetWidth) {
    targetWidth = inputWidth
  }
  if (!targetHeight) {
    targetHeight = inputHeight
  }

  // Crop/embed
  if (inputWidth != targetWidth || inputHeight != targetHeight) {
    if (fit === 'contain') {
      inputWidth = Math.max(inputWidth, targetWidth)
      inputHeight = Math.max(inputHeight, targetHeight)
    } else if (fit === 'cover') {
      if (targetWidth > inputWidth) {
        targetWidth = inputWidth
      }
      if (targetHeight > inputHeight) {
        targetHeight = inputHeight
      }
      inputWidth = Math.min(inputWidth, targetWidth)
      inputHeight = Math.min(inputHeight, targetHeight)
    }
  }

  return [inputWidth, inputHeight]
}

function resolveShrink(
  width: number,
  height: number,
  targetWidth: number | undefined = 0,
  targetHeight: number | undefined = 0,
  fit: 'cover' | 'contain' | 'fill' | 'inside' | 'outside',
  withoutEnlargement: boolean,
  withoutReduction: boolean
) {
  // if (swap && fit !== 'fill') {
  //   // Swap input width and height when requested.
  //   std::swap(width, height);
  // }

  let hshrink = 1.0
  let vshrink = 1.0

  if (targetWidth > 0 && targetHeight > 0) {
    // Fixed width and height
    hshrink = width / targetWidth
    vshrink = height / targetHeight

    switch (fit) {
      case 'cover':
      case 'outside':
        if (hshrink < vshrink) {
          vshrink = hshrink
        } else {
          hshrink = vshrink
        }
        break
      case 'contain':
      case 'inside':
        if (hshrink > vshrink) {
          vshrink = hshrink
        } else {
          hshrink = vshrink
        }
        break
      case 'fill':
        break
    }
  } else if (targetWidth > 0) {
    // Fixed width
    hshrink = width / targetWidth

    if (fit !== 'fill') {
      // Auto height
      vshrink = hshrink
    }
  } else if (targetHeight > 0) {
    // Fixed height
    vshrink = height / targetHeight

    if (fit !== 'fill') {
      // Auto width
      hshrink = vshrink
    }
  }

  // We should not reduce or enlarge the output image, if
  // withoutReduction or withoutEnlargement is specified.
  if (withoutReduction) {
    // Equivalent of VIPS_SIZE_UP
    hshrink = Math.min(1.0, hshrink)
    vshrink = Math.min(1.0, vshrink)
  } else if (withoutEnlargement) {
    // Equivalent of VIPS_SIZE_DOWN
    hshrink = Math.max(1.0, hshrink)
    vshrink = Math.max(1.0, vshrink)
  }

  // We don't want to shrink so much that we send an axis to 0
  hshrink = Math.min(hshrink, width)
  vshrink = Math.min(vshrink, height)

  if (fit === 'fill') {
    return [hshrink, vshrink]
  }
  return [vshrink, hshrink]
}


async function test() {
  sharp.concurrency(0)
  const start = 1
  const n = start + 100
  const width = 1000
  const height = 999
  const noPreshrink = false
  const xy = 50

  let ok = 0
  let failed = 0
  let err = 0
  for (const format of ['jpg', 'png', 'webp'] as const) {
    process.stdout.write(format + ' ')
    const img = sharp(
      await sharp({
        create: { width, height, background: 'white', channels: 3 },
      })
        .toFormat(format)
        .toBuffer()
    )
    for (const side of ['x', 'y', 'xy']) {
      process.stdout.write(side + ' ')
      for (const fit of [
        'cover',
        'contain',
        'fill',
        'inside',
        'outside',
      ] as const) {
        process.stdout.write(fit + ' ')

        const modes = ['on', 'noEnl', 'noRed']
        for (const mode of modes) {
          process.stdout.write(mode + ' ')

          const withoutEnlargement = mode === 'noEnl'
          const withoutReduction = mode === 'noRed'

          for (let i = start; i <= n; i++) {
            const targetWidth = side == 'x' ? i : side == 'xy' ? i : undefined
            const targetHeight = side == 'y' ? i : side == 'xy' ? xy : undefined

            try {
              let img2 = img.clone()
              if (noPreshrink) {
                img2 = img2.extract({ top: 0, left: 0, height, width })
              }
              const resized = await img2
                .resize({
                  width: targetWidth,
                  height: targetHeight,
                  withoutEnlargement,
                  withoutReduction,
                  fit,
                })
                .toFormat('jpg')
                .toBuffer()
              const meta = await sharp(resized).metadata()

              const [isW, isH] = resizeSize(
                width,
                height,
                targetWidth,
                targetHeight,
                fit,
                withoutEnlargement,
                withoutReduction,
                format,
                true,
                noPreshrink
              )

              if (meta.width === isW && meta.height === isH) {
                ok++
                process.stdout.write('.')
              } else {
                failed++
                process.stdout.write('\n')
                console.log({
                  format,
                  fit,
                  noEnl: withoutEnlargement,
                  noRed: withoutReduction,
                  file: { w: width, h: height },
                  trgt: { w: targetWidth, h: targetHeight },
                  real: { w: meta.width, h: meta.height },
                  calc: { w: isW, h: isH },
                })
              }
            } catch (e) {
              err++
              process.stdout.write('\n')
              console.log(e, {
                format,
                fit,
                noEnl: withoutEnlargement,
                noRed: withoutReduction,
                file: { w: width, h: height },
                trgt: { w: targetWidth, h: targetHeight },
              })
            }
          }
        }
      }
    }
  }
  console.log(`\ntests ok: ${ok}, failed: ${failed}, error: ${err}`)
}

This is a port from pipeline.cc only noPreshrink must be manually specified which can be determined as follows

// Try to reload input using shrink-on-load for JPEG, WebP, SVG and PDF, when:
//  - the width or height parameters are specified;
//  - gamma correction doesn't need to be applied;
//  - trimming or pre-resize extract isn't required;
//  - input colourspace is not specified;

Although I'm not sure if this can be known from the output of .metadata() maybe space: 'srgb' and hasProfile: false is enough? if not it would be good to add missing stuff to metadata output.

One thing which didn't work was fit: 'fill' there I had to swap the axes at the end of resolveShrink.

[pr0n1x2]

@ssendev Great job, thanks a lot. I have been looking for a solution for several days and already thought of digging into the source codes of C. But you did it before me and saved a lot of time. Thanks again.

I see so many requests from people to make this functionality part of the library, I don't understand why it's being ignored.