performance improvement

[micuat]

I've been testing if I can improve the performance https://glitch.com/edit/#!/p5-glitch-testing https://p5-glitch-testing.glitch.me/ (note that frameRate() is deceiving so I added a moving circle to visualize the framerate)

some changes:

• toDataURL seem to block everything on the page. It is better to use toBlob, but as it is asynchronous, functions like limitBytes cannot be used in the way how it is implemented. In the test I hardcoded it in the library...
• p5.loadImage seems not efficient so it may be better to use the base64 string directly into <img>. Then you can render it using p5.drawingContext.drawImage()... perhaps through a helper function.

The result is still not super smooth, but the difference is perceivable especially that the improved one is not blocking other processes (the circle's movement is jaggier in the original version)

Next step is to propose a PR but I don't know if it's possible without changing the API as explained above.

[ffd8]

Hey @micuat thanks for having a look into this and super interesting to learn that toDataURL blocks the page. A few questions:

• where in the example above are you asking it to do the glitching? I don't see a call for randomBytes() etc in the sketch.js (wanted to try and test other functions)
• curious why being async causes limitBytes() to stop working.. that should only effect the data after being loaded? Loading can use a callback function, so it should be possible to hold off with either checking if the glitch.image is defined or using the callback?
• while it's a clever way to place the image with drawingContext, I'm regularly using the glitched image as a texture in WEBGL mode = getting it back as a p5 loadImage() is really useful. Does that cause a hindrence?

Within P5LIVE, I still get quite good speeds using the existing and webcam – demo w/ circle

[micuat]

original version with the example I provided above (btw I updated p5.js to 1.4.0)

my edited version

so... I don't know if I made a fair comparison, I think 1) not only the blocking issue, toBlob seems performing better and 2) p5.loadImage seems taking a lot of time.

where in the example above are you asking it to do the glitching?

I directly added to the library: https://glitch.com/edit/#!/p5-glitch-testing?path=p5.glitch.edit.js L173

curious why being async causes limitBytes() to stop working.. that should only effect the data after being loaded?

I think limitBytes is affected to the old image, then the new image is loaded (because of async)

Loading can use a callback function, so it should be possible to hold off with either checking if the glitch.image is defined or using the callback?

or for example, limitBytes etc will be added to a stack, and as soon as the image is loaded (or with buildImage), these functions will be applied

while it's a clever way to place the image with drawingContext, I'm regularly using the glitched image as a texture in WEBGL mode = getting it back as a p5 loadImage() is really useful.

I think internally everything is a canvas... so there shouldn't be a problem using a native canvas as a webgl texture, but p5 might have some metadata that has to be passed on... I'm not sure.

Within P5LIVE

it seems working fine, but when I increase the speed of the circle (e.g., `(frameCount * 10) % width) I start to see hiccup.

[ffd8]

Playing with it in this remix, and I agree it is definitely getting a better/smoother framerates. It's clear that toBlob() async is a good idea to not block the page, while at the same time, it's nice to have a clear waterfall for loading, manipulating, and recompiling the bytes.

A few things I noticed while playing around:

• in that remix above, i could move limitBytes(), randomBytes() and buildImage() all to the draw (limitBytes() sets a class pref that's used from that point on).. so long as the image is never broken (which happens if limitBytes() is too low and hits crucial header bytes):

~~- unfortunately that method of using the drawingContext, crashes loudly (toggling on your edit, then moving mouse to very top causes the bad image = will happen all the time when glitching and needs to fail silently). It's producing the following error:~~

Uncaught DOMException: 
Failed to execute 'drawImage' on 'CanvasRenderingContext2D': 
The HTMLImageElement provided is in the 'broken' state.

~~To make it crash, toggle on the edited version, then click and drag mouse to top of canvas = setting limitBytes() to small enough number to kill header.~~

~~- attempted to solve issue above and the the texture issue by drawing it to a createGraphics() instead of directly to the p5 canvas.. however it still fails loudly.. once that can be solved, I could imagine this being implemented.~~

UPDATE: figured out a way to fail quietly! first implemented a tempImage.onerror to clear the glitch.htmlImage, but in the meantime, created a 2nd remix to test out tucking all those guts under the hood of the library = works. Nooow a new issue arrises.. one slowly falls behind in a framebuffer (~2sec)! haha = extra glitch... but it becomes a bit less realtime perhaps, due to the async. Solution is to probably try to clear that queue of onload taking place within the buildImage() or is it a lag with the fileReader() from toBlob()?