Support Offscreen Canvas, FileReader, Image, createImageBitmap APIs

[dtruffaut]

Please support:

• Offscreen Canvas
• ~~Blob~~
• FileReader: https://developer.mozilla.org/fr/docs/Web/API/FileReader
• Image: https://developer.mozilla.org/fr/docs/Web/API/HTMLImageElement/Image
• createImageBitmap: https://developer.mozilla.org/en-US/docs/Web/API/createImageBitmap

Just like a regular web worker in the browser would.

It would allow to redimension images and/or play with Tensorflow on the server, which is a secure environment VS the brower. (Browser is not secure because extensions act as proxies and can break/rewrite the CSP before serving the page to the user)

This feature is constantly asked since 2019 (3 years):

• june 2019: https://community.cloudflare.com/t/rendering-svg-using-canvas-with-cloudflare-workers/90952
• january 2021: https://community.cloudflare.com/t/blob-is-not-defined/233614/4
• october 2021: https://community.cloudflare.com/t/feature-request-support-offscreencanvas-api/315006

Of course there is the option of using Cloudflare Images, but the use cases are limited (only for resizing). Playing with composable primitives is prefered (image analysis, pixel per pixel, etc).

[kentonv]

Hi Denis,

We support Blob and File (mainly for use with FormData). I think we could add FileReader pretty easily, but I'm not sure it would actually be useful. It wouldn't actually be backed by a file, but rather data in memory (in a Blob). You still have to fetch that data from somewhere. It might make more sense to implement FileReader as a polyfill so you can have it do what you want in terms of reading the data from a source like Durable Object storage, KV, etc.

Canvas and Image are complicated APIs. Supporting them is conceivable but would be a very big project.

[dtruffaut]

DataUrl helps browser to display inlined images. It is useful for debugging, if you look at the second example below.

FileReader can be handy for snippets like:

// Read a blob and returns a dataUrl
// which is "basically" an inlined thing prefixed with its type
// data:[<mediatype>][;base64],<data>
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs

const toDataUrl = async (blob) => {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.addEventListener('error', () => reject(false), { passive: true });
    reader.addEventListener('load', () => resolve(reader.result), { passive: true });
    reader.readAsDataURL(blob);
  });
}

console.log(await toDataUrl(blob));

I also added createImageBitmap to the list. Here is an example of createImageBitmap, Offscreen Canvas, FileReader working together:

const resize = async ({ event, width }) => {

  // Get the image
  // While it would be possible to resize the image using this method, 
  // the resize is in reality performed later using OffscreenCanvas
  const image = await createImageBitmap(event.data.data);

  const height = Math.ceil(width * image.height / image.width);
  
  // Draw an Offscreen Canvas to convert into webp
  const canvas = new OffscreenCanvas(width, height);
  const contextOptions = {
    alpha: false,
    desynchronized: true,
    preserveDrawingBuffer: true
  };
  const ctx = canvas.getContext('2d', contextOptions);
  ctx['imageSmoothingEnabled'] = true;
  ctx['imageSmoothingQuality'] = 'high';
  
  // Resize happens here (image.width -> width ; image.height -> height)
  ctx['drawImage'](image, 0, 0, image.width, image.height, 0, 0, width, height); // source, destination
  
  // Convert into webp
  const webpBlob = await canvas.convertToBlob({ type: 'image/webp', quality: 0.8 });
  
  // Log the webp to debug
  const dataUrl = await toDataUrl(webpBlob);
  console.log(dataUrl);

  // Return the arrayBuffer for analysis, storage, ...etc.
  return webpBlob.arrayBuffer();
}

Having this kind of primitives is interesting, because in Machine Learning, developers often need to resize, grayscale, extract alpha... and perform various and lots of granular operations.

[jasnell]

Definitely agree with @kentonv here. FileReader doesn't really make a lot of sense in the workerd environment. Our File and Blob instances are always in-memory. FileReader wouldn't add much value, to be honest, and is difficult to justify. A polyfill wouldn't be difficult. Support for data URLs is definitely interesting, I think, and something we should look at but FileReader is not the only way we can produce those.

Canvas and Image would absolutely require a significant effort. Certainly something we can discuss but not likely to be on a priority list just yet. My key concern for those is that operations on those tend to be compute intensive, which works fine for standalone workerd but may not necessarily translate well into the multi-tenant edge environment where we have stricter cpu and memory limits enforced. Code that works in workerd might not be able to translate reasonably into the edge which could be problematic.

[CanRau]

canvas would be so amazing to generate dynamic og:images without depending on external services/APIs

[kentonv]

Hey all,

I know that support for canvas would be amazing! However, this is proposing an enormous project that would require a dedicated team and months (if not years?) of effort to build.

We prefer this issue tracker to stay focused on immediately actionable items, like bugs. We don't want to accumulate issues that are likely to stay open indefinitely. So, I am going to close this issue. However, I invite you to post discussions (under the discussion tab) to propose and discuss big ideas.

[klokan]

Relevant discussion continues at https://github.com/cloudflare/workerd/discussions/212

[TowhidKashem]

Hi Denis,

We support Blob and File (mainly for use with FormData). I think we could add FileReader pretty easily, but I'm not sure it would actually be useful. It wouldn't actually be backed by a file, but rather data in memory (in a Blob). You still have to fetch that data from somewhere. It might make more sense to implement FileReader as a polyfill so you can have it do what you want in terms of reading the data from a source like Durable Object storage, KV, etc.

Canvas and Image are complicated APIs. Supporting them is conceivable but would be a very big project.

for what it's worth I just had a need which could have been solved with FileReader, i need to read an image's EXIF data and to do so I need to load it in file reader before passing it to exif-js lib..