JavaScript 11ty.js templates support returning a buffer (sync `render`)

[monochromer]

I have issue with buffers returned from js template:

import { createCanvas, ImageData } from '@napi-rs/canvas';

export default {
  data: {
    permalink: '/assets/noise.png',
    tileSize: 64
  },

  render(data) {
    const tileSize = data.tileSize ?? 64;
    const black = new Uint8ClampedArray([0, 0, 0, 255]);
    const white = new Uint8ClampedArray([255, 255, 255, 255]);
    const result = new Uint8ClampedArray(tileSize * tileSize * 4);

    for (let row = 0; row < tileSize; row++) {
      for (let column = 0; column < tileSize; column++) {
        result.set(Math.random() > 0.5 ? white : black, (row * tileSize + column) * 4);
      }
    }

    const canvas = createCanvas(tileSize, tileSize);
    const context = canvas.getContext('2d');
    context.putImageData(new ImageData(result, tileSize, tileSize), 0, 0);

    return canvas.toBuffer('image/png');
  }
}

When i use async render function, it works fine. But with sync render i got broken image.

Code - https://stackblitz.com/edit/stackblitz-starters-jojfy7jh

[Ryuno-Ki]

I can reproduce (although not on Stackblitz because of how they build (using musl)).

mod in _getInstance of

https://github.com/11ty/eleventy/blob/b6622297b72e5b1d7d7d01f2c3008aacd39ec517/src/Engines/JavaScript.js#L76

is the default module you return.

Therefore

https://github.com/11ty/eleventy/blob/b6622297b72e5b1d7d7d01f2c3008aacd39ec517/src/Engines/JavaScript.js#L227

is called which produces a binary file (I have logged the output). The binary appears fine although my image viewer reports it as broken.

So I digged a little deeper:

xxd _site/noise.png > noise.hex  # Using async render
xxd _site/noise_broken.png > noise_broken.hex # Using sync render
diff --suppress-common-lines -y noise.hex noise_broken.hex
stat -c %s _site/noise.png
stat -c %s _site/noise_broken.png

The diff showed a longer file on the right side (the broken version) The stat compares 1628 vs. 2956 bytes. Therefore the sync call is writing more data into the Buffer.

Logging buffer.length shows a value around 1600. I have no clue where the additional bytes might come from.

[monochromer]

Problem in this code:

https://github.com/11ty/eleventy/blob/b6622297b72e5b1d7d7d01f2c3008aacd39ec517/src/Engines/JavaScript.js#L34-L40

https://github.com/11ty/eleventy/blob/b6622297b72e5b1d7d7d01f2c3008aacd39ec517/src/TemplateContent.js#L586

When function returns a promise, condition if (Buffer.isBuffer(result)) doesn't work. Further let rendered = await fn(data); get correct buffer.

With sync render inside normalize we get result.toString() with incorrect buffer.

[Ryuno-Ki]

I assume, it's designed for raw Buffer values.

Therefore I think the first step would be to add tests to https://github.com/11ty/eleventy/tree/b6622297b72e5b1d7d7d01f2c3008aacd39ec517/test/Util so we catch the current behaviour (and a failing test for what you intend to do).

A „quick” solution could be extending the signature of the normalize function to consider data as well. If the target isn't a text format (such as HTML), it shouldn't try to render a Buffer to a string. I have no clear picture in my mind on how this would play along with the other template engines.

[monochromer]

I see similar discussion here - https://github.com/11ty/eleventy/issues/2352.

I think buffer should not be special handled. If user use buffer, he knows how to work with one.

[Ryuno-Ki]

Hm, introduced back in v0.7.0: https://github.com/11ty/eleventy/commit/9072cbc73939b569202c066b7a0069c2d1614a8b

That was the release that introduced this Template Engine: https://github.com/11ty/eleventy/releases/tag/v0.7.0

I agree with you:

I think buffer should not be special handled. If user use buffer, he knows how to work with one.

[zachleat]

Hm! I’m okay adding this but if it’s not a 3.0 regression it will need to be moved to 4.0 as a breaking change

[zachleat]

In the mean time I’d welcome a PR that adds a configuration API opt-in to unlock this behavior in 3.0

[Ryuno-Ki]

Hi Zach,

yeah, it pretty much sounded like a breaking change to me. But then, we have one thing to look forward to in Eleventy v4 now 😇

[Ryuno-Ki]

@monochromer Do you have thoughts on how that API might look like? I'd imagine something like isBlob = false as function parameter (that could even lead to a non-breaking API!).

[monochromer]

Do you have thoughts on how that API might look like?

Maybe something like that:

// template.11ty.js
export default {
  // think about naming
  eleventyJavaScriptTemplateOptions: {
    blob: 'bypass' /* or `string` or function to transform buffer */

    /* or */
    postProcessRenderedResult: (result) => {}
  },

  data: {},

  render(data) {
    const buffer = getBufferSomehow();
    return buffer;
  }
}

[zachleat]

Shout out to related pagination-related discussion https://github.com/11ty/eleventy/discussions/4147

[zachleat]

Related to #2352