Deno support

[oscarotero]

Is your feature request related to a problem? Please describe.

Hi. First of all, thank for this great library. I have been maintaining a version of this code to work on Deno. You can see the repo here: https://github.com/lumeland/imagemagick-deno

Basically, it's a script to convert the CJS code to ESM and fix some minor issues.

In the latest two versions, the wasm file was moved to a different file. This is a good idea but makes it more difficult to maintain the Deno compatible version. As I can see, the code detects Deno as a Browser environment, so try to load the wasm file using XMLHttpRequest, which is an old API that Deno doesn't support.

error: Uncaught (in promise) RuntimeError: Aborted(ReferenceError: XMLHttpRequest is not defined). Build with -sASSERTIONS for more info.

Describe the solution you'd like

If the file was loaded using fetch() probably it would work on Deno.

Describe alternatives you've considered

As an alternative, maybe providing a way to pass directly the wasm content (instead of the file path) in Uint8Array (or any other format you consider) would be great. For example:

const response = await fetch(wasmFile);
const content = await response.blob();
await initializeImageMagick(content);

In this way the code would be more portable to other environments (like Deno).

Additional context

No response

[dlemstra]

It looks like the generated emscripten code contains an option to specify the content of the wasm binary and then that will be used instead of the location of the file. And I think the signature of the initializeImageMagick method can be changed to this:

initializeImageMagick(wasmLocationOrData?: string | Uint8Array | Buffer):

But I will need to experiment with that and see if that works.

And I am wondering if we could combine our efforts and automatically generate the deno library in this project? Not yet sure how we can do then you would not longer need to maintain that library and we can hopefully detect issues earlier. Maybe we could even run the unit tests in deno?

[oscarotero]

And I am wondering if we could combine our efforts and automatically generate the deno library in this project?

That would be great and I can help you, if you want. Due the code is written in TypeScript, I propose to make some tweaks in the code so it would work primary on Deno (and browser) and use https://github.com/denoland/dnt to build the Node version. I'm using this strategy in one of my projects and works great (https://github.com/oscarotero/keep-a-changelog/blob/master/build_npm.ts).

[dlemstra]

I could use some help with this but I would prefer to go from node to deno instead of the other way around. This project use vite and a lot of its tools for the build process so I would like to create the deno package from the existing code in this project. And I also wonder if the the dist/index.mjs file is not already compatible with deno.

[oscarotero]

Okay, understood.

Deno works very similar to browsers (it uses Web standards as much as possible) but it also has native support for TypeScript. In Node the imported files are resolved magically. For example:

import magick from './magick'

This can import the file ./magick.js, ./magick.mjs, ./magick/index.js, ./magick.ts, ./magick/index.ts ...etc. It depends on the real filename so it has to check different combinations before load the file. Deno works like a browser so you have to pass the real filename including the extension: import magick from './magick.ts'. My script to convert the code from ./src folder to Deno only adds these extensions to the imported files.

The other thing was to copy the module that had the wasm code, including a ESM export to be imported correctly in Deno (because it was only UMD). In the new versions, this step is broken, because the wasm file is loaded dynamically in the browser environment, which uses XMLHttpRequest (not implemented by Deno because they recommends to use Fetch).

If want to know which web standards are supported by Deno, take a look to this cheatsheet (under the Web APIs section). As you can see, only Fetch is supported but there are different ways to instantiate WebAssembly modules.

[dlemstra]

I was able to implement loading the wasm data from a Buffer or Uint8Array and this will become available in the next release. This will be committed later tonight or tomorrow because this also needs a change in Magick.Native.

Are you aware that the content of the dist folder has changed since version 0.0.16 and that there now is a single file with all the exports and both an index.mjs and index.umd.js file. Would it be possible to use these files in a deno package?

[oscarotero]

Are you aware that the content of the dist folder has changed since version 0.0.16 and that there now is a single file with all the exports and both an index.mjs and index.umd.js file. Would it be possible to use these files in a deno package?

I just tried index.mjs but got this error:

error: Uncaught ReferenceError: __DENO_NODE_GLOBAL_THIS_0_38_0__ is not defined
    at file:///Users/oscarotero/imagemagick-deno/node_modules/@imagemagick/magick-wasm/dist/index.mjs:1:18

I would also prefer to use the TypeScript code in Deno, in order to have proper types.

I also tried your latest commit on Deno (after transforming the code using my script):

console.log("Initialize");

const wasmFile = import.meta.resolve("../deno/src/wasm/magick_native.wasm");
const file = await fetch(wasmFile);
const wasm = await file.arrayBuffer();

await initializeImageMagick(wasm);

console.log("Read");
const data: Uint8Array = await Deno.readFile("test/unsplash.jpg");

await ImageMagick.read(data, async (img: IMagickImage) => {
  console.log("crop");
  img.crop(1000, 1000);

  console.log("resize");
  img.resize(200, 100);

  console.log("write");
  await img.write(
    (data: Uint8Array) => Deno.writeFile("test/unsplash-blur.png", data),
    MagickFormat.Png,
  );
});

And the code seems to work (it read, crop and resize the image) but fails in the img.write:

Initialize
Read
crop
resize
write
error: Uncaught (in promise) TypeError: str.charCodeAt is not a function
    at WasmLocator.lengthBytesUTF8 (file:///Users/oscarotero/imagemagick-deno/deno/src/wasm/magick_native.js:8:15138)
    at _withNativeString (file:///Users/oscarotero/imagemagick-deno/deno/src/internal/native/string.ts:23:22)
    at _withString (file:///Users/oscarotero/imagemagick-deno/deno/src/internal/native/string.ts:38:10)
    at new NativeMagickSettings (file:///Users/oscarotero/imagemagick-deno/deno/src/settings/native-magick-settings.ts:69:7)
    at MagickSettings._use (file:///Users/oscarotero/imagemagick-deno/deno/src/settings/magick-settings.ts:95:22)
    at file:///Users/oscarotero/imagemagick-deno/deno/src/magick-image.ts:2356:24
    at Function.use (file:///Users/oscarotero/imagemagick-deno/deno/src/internal/pointer/pointer.ts:27:14)
    at file:///Users/oscarotero/imagemagick-deno/deno/src/magick-image.ts:2355:15
    at file:///Users/oscarotero/imagemagick-deno/deno/src/internal/exception/exception.ts:46:22
    at Function.use (file:///Users/oscarotero/imagemagick-deno/deno/src/internal/pointer/pointer.ts:27:14)

[dlemstra]

I was able to run this project with deno. I have added an example to this project to show how I did that. And I have also added a build step that will run a simple example that reads and writes an image using deno.

Your code is failing because I made another breaking change in the main branch that swaps the arguments of the img.write method.

[oscarotero]

Okay. I'd rather to use directly the TypeScript code in Deno (for better debugging) but if the mjs file works, that's fine.

My version is published on lume.land/x: https://deno.land/x/[email protected].

If you are going to maintain the Deno version, probably I should deprecate this package and create a new one pointing to this repo?

Or perhaps it's possible to edit the current package to use this repo.

[oscarotero]

Not related with this, but when I try to convert images to AVIF format, the browser cannot display them (tested on Firefox and Chrome). It says the image cannot be shown because it contains errors.

[dlemstra]

Not sure how I want to move forward with Deno support. At least I have a proof of concept now on how someone could get it working but a proper module would probably be the right idea. Maybe a separate build step that makes this project available as a module?

I cannot reproduce your avif issue. Maybe create a separate discussion for that? And make sure that you are using the latest code on main. I recently fixed a bug in the aom encoder build so that might be causing what you see.

[oscarotero]

In deno.land you can register a new module choosing a directory inside a GitHub repo. For example, my version (https://deno.land/x/imagemagick_deno) contains the code inside the /demo/ folder of my repo (https://github.com/lumeland/imagemagick-deno/tree/main/deno). Your build code could export a Deno version in a subdirectory and register the module from that. Then, it uses git tags for new versions. In Deno is common to use the mod.ts instead of indext.ts as the filename for the main module, so you can create a different entry points for Deno.

Another think is about cache. Deno doesn't have a node_modules folder or a npm install. It download and cache the modules the first time they are used. If the wasm code is loaded from fetch, it won't be cached so it must be downloaded everytime the file is runned. Related: https://github.com/denoland/deno/issues/5987

Edit: I could manage to cache the wasm code using the Web Cache API, that is supported by Deno. Here's the script: https://github.com/lumeland/imagemagick-deno/blob/main/deno/mod.ts

---

And about avif, I'll do more tests and wil open a new issue if I cannot fix it. thanks!

Edit: The latest version works fine generating avif formats. 🎉

[pseudosavant]

Can I just say thank you to you two for working on this? I'm dying to using imagemagick via wasm + Deno. There is no way I want to call out to a local bin when accepting user file input.

[shynome]

it seems already work? (by use npm: specifiers) so great projects!

import {
  ImageMagick,
  initializeImageMagick,
  Magick,
} from "npm:@imagemagick/magick-wasm";

await initializeImageMagick();
console.log(Magick.imageMagickVersion);
const buf = await fetch(new URL("./download.jpeg", import.meta.url)).then((r) =>
  r.arrayBuffer()
);

let x = await ImageMagick.read(new Uint8Array(buf), async (img) => {
  return { w: img.width, h: img.height };
});
debugger;