deno-canvas canvas & 2D context & image objects do not match the browser

[backspaces]

This is from a deno issue, https://github.com/denoland/deno/issues/18919 The response was:

that module is a third party module maintained by the community, not an official module maintained by Deno. I'd recommend opening an issue on the repository of the module.

Here's the issue & thanks for the great project!

---

A few experiments show that the deno-canvas's canvas, context, and image objects do not match the browsers. This makes it difficult to impossible to have interoperability between deno and browsers, a goal of many of us.

Some examples:

// copy/paste these into a deno repl (deno repl -A)
import {
    createCanvas,
    loadImage,
} from 'https://deno.land/x/[email protected]/mod.ts'

// creaate image, canvas & ctx
const imgUrl = 'https://code.agentscript.org/models/data/redfish.png'
const img = await loadImage(imgUrl)
const can = createCanvas(200, 200)
const ctx = can.getContext('2d')

// 1: ctx has a flawed canvas object:
typeof ctx.canvas // OK, "object"
// but it isn't!
ctx.canvas.width // TypeError: Cannot read properties of null!
Object.keys(ctx) // shows no canvas object!
ctx.canvas = can // TypeError: Cannot assign to read only property 'canvas' of object

// 2: ctx has width & heigh while in browser ctx.canvas has the width & height
// see above

// 3: img width & height are functions, not properties as in browser
img.width // [Function: Image$width]
img.width() // works: 512

// 4: prototype programming fix works for ctx.canvas, but alas not for img
const ctxObj = {
    canvas: can,
}
Object.setPrototypeOf(ctxObj, ctx)
ctxObj.canvas  // OK
ctxObj.canvas.width  // OK

const imgObj = {
    width: img.width(),
    height: img.height(),
}
Object.setPrototypeOf(imgObj, img)
ctx.drawImage(imgObj, 0, 0, can.width, can.height) // TypeError: k.width is not a function
ctx.drawImage(imgObj, 0, 0, imgObj.width, imgObj.height) // Ditto

---

I believe the man problem is Image using functions for width/height, we might be able to work around the rest. Or do you have an insight how we might work around that issue too?

Thanks!

[DjDeveloperr]

Image object in here is quite different from Web Image object in browsers, here it is an object that wraps SkImage. It will be difficult to change its API now. As a workaround, we can make a new web-compatible Image object that is accepted along with Skia Image object.

Other issues that were mentioned seem like a regression when the underlying library was updated. They’ll be an easy fix.

Btw, have you considered using skia_canvas (https://deno.land/x/skia_canvas)? Which is much more accurate and faster. It is based on Skia too and is web compatible. Unless the goal is to run on serverless or other environments that do not support FFI/Native Libraries, I recommend using skia_canvas (it’s also authored by me).

[backspaces]

Hi, and thanks for the quick and informative response!

Let me briefly sketch out my work and how the compatibility between Deno and the browser and workers matter.

My Agent Based Modeling system uses a very strict Model/View split. Models deal with data and their transforms while the view provides visual and interactive (MVC) views of the model.

Here are three examples: https://code.agentscript.org/views2/tsp.html https://code.agentscript.org/views25/roads.html https://code.agentscript.org/views3/avalanche.html

The Models can run almost everywhere: browser, workers, Deno; in 2D, 3D and on Maps. Deno comes in in two ways: yet another place to run Models storing their data and images in files .. but also being our main test system now. It's FAST, reducing our earlier testing from well over a minute to 6 seconds!

Models do not create images, that's for Views. BUT Models use images as Data! GIS tiles, elevation data and so on. And indeed we're interested in Deno as an off-line data transformer, getting OpenStreetMap (OSM) data and crunching it. The data can be images.

So managing all this would be easier if deno-canvas were compatible.

It seems that I can work around most problems. Image, I think, is the biggie. We currently have to manage the Dom Image & canvas vs the Worker imageBitmap and Offscreen canvas which is fairly simple. But deno-canvas Images having functions for width/height is pretty messy.

So any help would be great!

[backspaces]

OK, I've now got some slightly ugly workarounds that let me run all 24 of my demo AgentScript Models (not Views .. yet!).

OTOH, yes it would be nice to have a truly compatible deno image, canvas, and 2d context. For me Image having functions for width/height and canvas context not having the ctx.canvas work properly are the main issues in terms of getting stuff to run. I haven't tested deeply .. for example does the same (data) image produce the same results in the browser and deno.

One thing I'm starting to use in the browser is the ImageBitmap and the OffscreenCanvas, both required for browser workers. Kinda nice!

Thanks again for the help and insights.

[DjDeveloperr]

We have web-compatible Canvas, Context2D and Image objects in the Skia Canvas project. The API is nearly same, and it only has more capabilities than deno-canvas rather than being less. Can I provide any help in getting Skia Canvas work according to your needs instead?

If not, I can proceed with improving web compatibility of this module so that it can fit your use case.

Some more info: While deno-canvas is a port of canvaskit-wasm, that is Wasm compiled Skia-based implementation of HTML Canvas, originally made for Node.js and the browser, it has many inconsistencies with the web API which are harder to overcome. On the other hand, Skia Canvas was made from scratch using Skia native library and close attention was paid to web compatibility.

[backspaces]

OK, let me use Skia Canvas a bit to see if it resolves the problems I had with deno-canvas. Thanks again!

[backspaces]

Oops, I may be doing this wrong.

First I ran deno upgrade .. I was already current: master: deno upgrade Looking up latest version Local deno version 1.33.1 is the most recent release

Then I tried importing all of skiaCanvas, or just Canvas & Image. I.e either of: import { Canvas, Image } from 'https://deno.land/x/[email protected]/mod.ts' import * as skiaCanvas from 'https://deno.land/x/[email protected]/mod.ts'

Both failed. I tried both with just -A and with -A --unstable flags: deno repl -A deno repl -A --unstable

master: deno repl -A --unstable 
Deno 1.33.1
exit using ctrl+d, ctrl+c, or close()
> import * as skiaCanvas from 'https://deno.land/x/[email protected]/mod.ts'

Uncaught Error: Could not open library: Could not open library: dlopen(/Users/owen/Library/Caches/deno/plug/https/github.com/dde09b5bbcf801e9d3dbb0bf6276f9d048d6643f6b34855bc184ebffa990fd30.dylib, 5): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
  Referenced from: /Users/owen/Library/Caches/deno/plug/https/github.com/dde09b5bbcf801e9d3dbb0bf6276f9d048d6643f6b34855bc184ebffa990fd30.dylib (which was built for Mac OS X 12.6)
  Expected in: /usr/lib/libc++.1.dylib

    at new DynamicLibrary (ext:deno_ffi/00_ffi.js:440:46)
    at Object.dlopen (ext:deno_ffi/00_ffi.js:577:10)
    at dlopen (https://deno.land/x/[email protected]/mod.ts:145:15)
    at eventLoopTick (ext:core/01_core.js:166:11)
    at async https://deno.land/x/[email protected]/src/ffi.ts:961:10

And here's the other import:

deno repl -A --unstable 
Deno 1.33.1
exit using ctrl+d, ctrl+c, or close()
> import { Canvas, Image } from 'https://deno.land/x/[email protected]/mod.ts'

Uncaught Error: Could not open library: Could not open library: dlopen(/Users/owen/Library/Caches/deno/plug/https/github.com/dde09b5bbcf801e9d3dbb0bf6276f9d048d6643f6b34855bc184ebffa990fd30.dylib, 5): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
  Referenced from: /Users/owen/Library/Caches/deno/plug/https/github.com/dde09b5bbcf801e9d3dbb0bf6276f9d048d6643f6b34855bc184ebffa990fd30.dylib (which was built for Mac OS X 12.6)
  Expected in: /usr/lib/libc++.1.dylib

    at new DynamicLibrary (ext:deno_ffi/00_ffi.js:440:46)
    at Object.dlopen (ext:deno_ffi/00_ffi.js:577:10)
    at dlopen (https://deno.land/x/[email protected]/mod.ts:145:15)
    at eventLoopTick (ext:core/01_core.js:166:11)
    at async https://deno.land/x/[email protected]/src/ffi.ts:961:10

Running without --unstable gives this:

master: deno repl -A 
Deno 1.33.1
exit using ctrl+d, ctrl+c, or close()
> import { Canvas, Image } from 'https://deno.land/x/[email protected]/mod.ts'

Uncaught TypeError: Deno.dlopen is not a function
    at dlopen (https://deno.land/x/[email protected]/mod.ts:145:15)
    at eventLoopTick (ext:core/01_core.js:166:11)
    at async https://deno.land/x/[email protected]/src/ffi.ts:961:10

I bet I'm missing something! :) Any advice? I like skia's api etc, looks promising, but gotta get over this hump. Thanks for your patience!

[backspaces]

I noticed that the deno build was for mac os x 12.6 which I don't have due to having an older iMac (late 2014). That could be the problem. dde09b5bbcf801e9d3dbb0bf6276f9d048d6643f6b34855bc184ebffa990fd30.dylib exists but possibly cannot load? I'll upgrade my laptop and see if things work there.

[backspaces]

Yup, when I imported the skia_canvas modules with my newly installed Ventura macOS on my M1 macbook, the imports worked. But my dev iMac is too old for an upgrade higher than it's current BigSur, sigh.

Do you know if there are versions of skia_canvas that would work on BigSur?

[backspaces]

I'm afraid there is no simple way to get a JS canvas and Image with either deno_canvas or with skiaCanvas. Skia seems the most likely to succeed but it doesn't import correctly in workers!

I'll see if skia avoids the list of problems up top. If so, my choices are:

• Use skia due to the standard API and lack of the other oddities. But not allow use in workers.
• Use deno with several ugly workarounds, but be able to run in workers.

I'll try skia for a couple of days and hope we can resolve the worker import failures.

Let me know what you think.