Updated to nan 2.4.0

[ReneHollander]

I updated to nan 2.4.0 to allow the usage of newer node versions. The examples work as expected on my Raspberry Pi 3, but there are two things that I was unable to solve:

• openvg.cc:1127: I really have no idea how to fix that or if a fix is even needed (just remove it)
• openvg.cc:1712: The Unwrap causes a wierd compiler error that I also can't solve

I hope that we can fix the last to problems and merge this pull request. It is otherwise impossible to use this package in newer node versions.

[ReneHollander]

I think I fixed the first issue on line 1127 in openvg.cc, but the unwrap is still causing problems.

[skerit]

Hey @ReneHollander, I'm trying your fork on my Raspberry Pi 3, using node v6.9.0, but the samples fail with this error:

node: ../src/egl.cc:77: void egl::Init(): Assertion `success >= 0' failed.
Aborted

Any idea what the problem could be?

[ReneHollander]

Did you make sure that X is not running?

You can try to debug the app with gdb. For that you need to create a debug build of the library with node-gyp rebuild --debug or a similar command. If you set a breakpoint in the egl::Init function you can step through and output the actual value of the success variable and check the EGL documentation to find out what is not working.

[skerit]

Right, I did have X running.

I've disabled it, but now I get failed to add service - already in use. Some issue with the GL driver? Which driver am I supposed to use?

[ReneHollander]

I have never seen this error. You don't need a special driver. Only the videocore libraries that are installed in /opt/vc. Somewhere in the /opt/vc directory are examples. Can you try to compile and run those?

[skerit]

I had the experimental GL drivers installed, but couldn't disable them for some reason. Creating a clean image did the trick: the examples are running fine :)

I didn't realize the node-openvg-canvas module also had to compile certain things (like freeimage), I'm guessing that's not on your todo-list? :)

[ReneHollander]

My plan was to get luismreis/node-openvg-canvas running. Here is my current effort: ReneHollander/node-openvg-canvas). I don't know if it works, because I only worked on it in september and can't remember if got it running.

[skerit]

Thanks a lot! ~~Most examples run smoothly, only the image examples are throwing some errors like this is not a typed array and ctx.getImageData is not a function.~~

I've forked your fork and re-added the getImageData and putImageData methods again, most examples seem to work now.

What was your original use case for this anyway? And why did you stop?

I'm gonna try to use this with https://github.com/jean343/Node-OpenMAX I've already added some proof-of-concept code here: https://github.com/skerit/omaxtest

[ReneHollander]

I was not able to render an OpenMAX video to an OpenVG image correctly and gave up. After that I tried to get skia running on the Raspberry Pi, but I did not get far with it.

[skerit]

Ah interesting, that's what I'm trying to do, too. I've got the video playing, kind of. It is in black and white, but that's because I'm just dumping the raw decoded pixels onto the screen.

[ReneHollander]

The problem I had, was that I could not scale the OpenVG image of the video without the video disappearing. It works with OpenGL ES, and that is why I tried to get skia running. It uses GLES to accelerate the drawing.

[skerit]

How did you do the scaling? I'm guessing rescaling the image using the freeimage module is too resource intensive?

Did you do all of this in javascript or C++? I'd love to take a look at the code you wrote to output video :)

[ReneHollander]

You can directly stream the output of the VideoDecode OpenMAX component to an EGL Image that gets rendered with OpenVG. This will be handled entirely by the VPU and will put not stress on the CPU. You can than scale and transform an image with the vgScale and vgTransform commands, but by applying a scale, the video is no longer shown.

I don't know if I still have the code, but the proof of concept was done in C++. Most of the code is from the book "Raspberry Pi GPU Audio Video Programming" by Jan Newmarch. The book explains everything about OpenMAX, OpenVG, OpenGLES and all possible interactions between those APIs.

[skerit]

Well, I had some luck scaling the openmax buffers like this:

var img,
    ar = new Uint8Array(data.buffer);

img = vg.createImage(vg.VGImageFormat.VG_sARGB_8888_PRE, 1920, 1080, vg.VGImageQuality.VG_IMAGE_QUALITY_FASTER);

// Unfortunately: setting the image data takes 80ms
vg.imageSubData(img, ar, 1920, vg.VGImageFormat.VG_sARGB_8888_PRE, 0, 0, 1920, 1080);

ctx.clearRect(0 ,0 , 1920, 1080);

mm = m.create();
vg.getMatrix(mm);
var savMatrixMode = vg.getI(vg.VGParamType.VG_MATRIX_MODE);
vg.setI(vg.VGParamType.VG_MATRIX_MODE, vg.VGMatrixMode.VG_MATRIX_IMAGE_USER_TO_SURFACE);
vg.loadMatrix(mm);

// Scaling and drawing
vg.scale(0.5, 0.5);
vg.drawImage(img);

// Restore settings
vg.setI(vg.VGParamType.VG_MATRIX_MODE, savMatrixMode);
vg.loadMatrix(mm);

canvas.vgSwapBuffers();
vg.destroyImage(img);

But yeah, this does put stress on the CPU, setting the image data takes 80ms. Setting the pixels directly using writePixels is fast, but that bypasses the scale.

I thought OpenMax had some kind of eglimage class, but I can't seem to make use of that.

[ReneHollander]

Here is my code that renders a video directly to an EGLImage. It is a mess, but if you remove all the skia stuff, and look at the commented code, you may be able to get it running again. But as I said, as soon as I try to scale or transform, it doesn't work anymore...

screen.zip