5-6 seconds - or more, before model load and client (user) can use the page - how to improve?

[bongobongo]

I'm currently testing this interesting nsfwjs module on a live server.

I activate this code: pdModel = await nsfwjs.load('model/');

via a call to an async function triggered by the onLoad event. The 6 model files are fetched from the disk and is using approximately 960ms to load (I see this using Chrome inspector - Network tab). After this I have to wait approx. 5-6 seconds before I get any response from the page. From what happens in the Memory tab in Inspector, it looks like the nsfwjs load is using this time to do something.

After the 5-6 seconds. I can select an image. When that is done, this code (in another async function): var img = new Image(); img.src = document.getElementById('img_url').src; var predictions = await pdModel.classify(img); console.log('Predictions', predictions);

is using about 1,5 - 2 seconds to finish. This is acceptable time for me. This time usage of about 1.5 -2 seconds - is consistent.

But the usage of 5-6 seconds nsfwjs use to do something - before anything else can happen in the page (e.g. select an image to analyze), is quite a long time - for an end user which want to do something as soon as possible.

So... is there a way to speed up the time nsfwjs uses to finish this part: pdModel = await nsfwjs.load('model/'); and whatever it does before the page is ready for handling user triggered events in the page?

Could the preparation work done by nsfwjs (these first 5-6 seconds) maybe be split up in 2 parts, and then do half the job - onLoad, and then the second part - which is triggered to happen when an image actually is selected by the user?

[GantMan]

Try updating your version of Tensorflow maybe?

[bongobongo]

I'm using this: https://cdn.jsdelivr.net/npm/@tensorflow/[email protected] Is there a newer version available?

[GantMan]

You should definitely keep track of what CDN packages are available

https://cdnjs.com/libraries/tensorflow

[bongobongo]

Thanks for that information. Now tested with latest tfjs 1.1.0. Same results. Any idea on how to not make the client (user) wait "so long" (5-6 seconds) before he can interact with the page (see suggestion in the end of thread start)?

[GantMan]

Hrmmm. I really don't know. Is your code open source? I can take a look if so.

[bongobongo]

Thanks for that.

You can test it from here Explanation is in the page: nude-test-await

But this should be no surprice. The same delay can be experienced on this demo page: https://nsfwjs.com/ Same delay there - even if the model files are fetched from disk.

Here is the relevant information on how the page was set up:

`

const nsfwjs = require('nsfwjs'); var pdModel;

async function loadAwait() { pdModel = await nsfwjs.load('model/'); }

async function analyzeImageAwait(){

if (pdModel){	
	var img = new Image();
	img.src = document.getElementById('img_url').src;
	var predictions = await pdModel.classify(img);
	console.log('Predictions', predictions);
    }

} ` in the body element I use this: onload="loadAwait()"

on the server I have these sub directories with the following files: model/ files: group1-shard1of6 group1-shard2of6 group1-shard3of6 group1-shard4of6 group1-shard5of6 group1-shard6of6 model.json

nsfwjs/ files: bundle.js tf.min.js.map [email protected]

Except for this the main file only contain an input file field where I can select an image on the computer, and some javascript code which copy the relevant file information to the img.src with id img_url - like this: function img_pathUrl(input){ $('#img_url')[0].src = (window.URL ? URL : webkitURL).createObjectURL(input.files[0]); setTimeout(analyzeImageAwait, 1000); }

But as allready mentioned the delay of 5-6 seconds happens before I can click the file button which activate the img_pathUrl function above.

[GantMan]

So this is going to sound strange. Neither NSFWJS or your page have this delay for me. When I load your page, the first time, it downloads the model and then I was immediately able to access the UI.

The second time I access the page, it reconfirms the cache for a few seconds, but the very second your loader goes away, I'm able to click.

Perhaps it's a browser/computer difference? Would you be willing to google/run some other TensorflowJS examples and see if this happens across the board for you? Perhaps it's when the model is loaded into memory? Instant for some, slower on other computers?

[bongobongo]

What is the load time in seconds (which shows in blue text in page after loader hides) when the model files loads from disk, when you use this page? nude-test-await

[GantMan]

Cold start

Second page load

For me, the spinner shows me that the UI would be inaccessible. But the second it goes away I can click.

Side note

As far as I can tell, this is verifying local cache and loading the model into memory.
I might train a smaller model soon and see if I can get it nearly as accurate. If so, that will be helpful with these kinds of times. As you can guess, training models can take a long time.

[bongobongo]

My point is that the load time takes a lot of time - and that this might make it awkward to use in a production environment. In my web application which is still under development, the user will want to interact rather quickly with the UI, not wait 5 - 20 seconds before he can select an image, or do something else in the page.

The thing is, there is a huge difference between time to fetch the model files from disk, which on my computer takes only 400 - 600 milliseconds, and the total time the load stuff actually work (as long as the loader spins). After the loader spinner stops spinning (after load process finish) the page is accessible also on my computer.

My question is related to if there is a way to make the load time (except for the 400 - 600 milliseconds for the files to be read from disk cache) shorter, or if possible to divide this work into e.g. two parts. If the latter (two parts) - then one part could be activated when user selects the image, and the other part when he submit the form. This way - the wait, could be non-problematic for end user - in my opinion. I say could because the time seems to vary between browsers (and platforms, e.g. mobile), and even between different page loads in the same browser.

Another way to speed things up, could be to NOT analyze drawing and neutral. For my use it would be enough to test for a numeric value for Porn, Sexy and Hentai content. Is it possible in some way to force the module to a) not load the model files for Drawing and Neutral, and b) not test for these values?

Or what about this idea. Open a new hidden browser window via Javascript, then load e.g. a similar html page as this page, and then load the model there without blocking usage of the actual page. Then when the user click submit, or select an image in the actual page, the img src is copied into javascript in the tab with the model - then do the analyzing there, and return the answer back to the actual page.

Today I also tested the page, I gave you the link to, on my Samsung Galaxy Note II - on a Wi-Fi connection. It took 121 seconds to finish the first load, and 181 seconds to finish the second load. When selecting an image - and waiting for a couple of minutes - no predictions where output. Need to do more tests here - since I have no clue why it was so slow on second load, and why no predictions where output (UPDATE: the day after I tested this again. It actually works, but needed some more time to finish - see my next comment below).

In Internet Explorer (version: 11.0.9600.17728) the model does not load. The error is from the [email protected] file, where debugger report this error: "'Symbol' is undefined", where the debugger highlighted this code:

I also observe the 'Symbol' error in IE, mentioned above - when testing the of NSFWJS demo page in IE.

[bongobongo]

GantMan There is something called Web Workers in Javascript. Is it possible to load the model in a web worker, and then make the model load behind the scenes, so to speak, so the user does not have to wait until the model has loaded before the user can start to interact with the web page.

When I have this in the worker.js importScripts('[email protected]'); importScripts('bundle.js');

Then I get these errors in the console: Uncaught Error: Could not find a global object at engine.ts:829 at engine.ts:818 at engine.ts:836 at [email protected]:2 at [email protected]:2 at worker.js:13

If I click the "at engine.ts:829, it reports error at this line: throw new Error('Could not find a global object');

So, do you know if it is possible to run the nsfwjs module from within a web worker? More about web workers here: Multithreading Java script

[GantMan]

Personally, I haven't worked with Web Workers just yet, but some searching found this discussion which I think should be noted: https://github.com/tensorflow/tfjs/issues/102

[bongobongo]

Personally, I haven't worked with Web Workers just yet, but some searching found this discussion which I think should be noted: tensorflow/tfjs#102

Thanks for that information, just what I needed. Have tested it, and Web Workers do work with NSFWJS (in Opera and Chrome). Which mean that the model can load in a background process, while the client can start interacting with the user interface of the web page immediately. But could you please tell me if it is possible in some way to not use the model for Drawing and Neutral. That is, just download (load) the model files for analyzing image for Porn, Sexy and Hentai? Thereby save time by not having to download the two mentioned model files (Drawing and Neutral) to the client, and hopefully awoid that the model.classify() use time on testing for Drawing and Neutral.

An even better solution would be to create separate Web Worker for analyzing if Porn, Sexy, Hentai (and the others if one want that: Drawing and Neutral). The first Web Worker could initiate the other Web Workers after the nsfwjs is created, but before loading the model for testing for Porn. This way the other workers would fetch e.g. bundle.js and [email protected] from cache (second worker would then fetch model file(s) for anlyzing Sexy, the third worker would fetch file(s) for analyzing Hentai (and so on). I believe such an approach would speed up the load time drastically on mobile devices, as well as speeding up load time in general on other devices - since each worker is working in it's own separate process at the same time. The process of analyzing frames (images) from a video, should also be a lot faster - using Web Workers.

If this can be done, when can we see new model files and new other files on github - which makes it possible to load e.g. separate model for Porn, and analyze just for Porn (and same for the other types (e.g. Sexy, Hentai, etc) - in a separate Web Worker?

For this to work in the Web Worker, the Web Worker must support Offscreen Canvas. Not all web browsers support this yet, but it would be easy to test in a web worker js file to report if the feature is supported on the client browser, and if not - do not use use web workers. Here one can see which browsers (versions) support Offscreen Canvas.

I mentioned in my previous comment above, that I could not make the NSFWJS work in my mobile (Galaxy Note II) using Chrome. That was not correct. Did a new test yesterday, and found that the image analyzing did work on the mobile device. I did not wait long enough the first day I tried this. But it took about 3 minutes and 30 seconds just to analyze the image (get the predictions). Approximately the same time it took to load the model.

[GantMan]

Unfortunately the model is all in one and would not be made smaller by removing classifications.

However! I'm nearing the end of training a smaller and nearly as accurate model. Expect that soon. It should speed everything up

Re: https://github.com/GantMan/nsfw_model/issues/20#issuecomment-487354103

[bongobongo]

Look forward to test that. But, if you trained a model to recognize only Porn, Sexy and Hentai images, or just Porn, would that model (the files) not be smaller than a model trained to recognize (predict) Porn, Sexy, Hentai, Drawing and Neutral?

[GantMan]

So the model is architected first and only the final layer is affected in size by number of possibilities.

Now, fewer possibilities could be supported by a smaller model architecture, but that would require building a smaller model and retraining. Aka lots of time. It's hard to remove those parts without throwing the whole model out of balance. As far as I know it's not even a practice. But it should be! That would be cool.

Anyhow, best case is to get this new smaller model I'm making and do a conditional on the predictions.

I fully believe model optimization will become a very real thing in the next 5 to 10 years. Just not a simple solution.

[danieldaeschle]

Is there noe way to run it non-blocking? Which means it should run async in background with a callback when it finishes...

I have the same problem. I want to create a chrome extension. But waiting on every site 5-6 seconds is just annoying.

[bongobongo]

danieldaeschle: You can load the model and analyze the images in a non blocking mode. Just create the NSFWJS and load the model in a web worker - then it will run in a separate thread on the client. I have tested this, and it works in latest versions of Chrome and Opera. This way the web page UI is not blocked by the loading procedure of NSFWJS.

@GantMan , could you please confirm if the suggestion below is technically possible: After figuring out that I can load the NSFWJS module in a web worker, my next thought was if it was possible to load the data from the different model files using separate Web workers, and just send the loaded data (array, object) or whatever is loaded in a web worker, to the main web worker (the one you want to use for analyzing the images). This way the data from the model files could be loaded in parallell - and could probably minimize the total load time - which would be especially beneficial if the client is on a mobile device, but should be faster on all devices. Note: It is possible to send e.g. arrays and objects from a worker back to main script (page) and from that script to another worker.

[danieldaeschle]

@bongobongo like i already said, i want to create a chrome extension which does not support web workers. i just tried it :(

[bongobongo]

@bongobongo like i already said, i want to create a chrome extension which does not support web workers. i just tried it :(

@danieldaeschle , have you looked into this: https://bugs.chromium.org/p/chromium/issues/detail?id=357664#c3

[danieldaeschle]

I checked both. Nothing works. They are 5 years old...

bongobongo [email protected] schrieb am Di., 30. Apr. 2019, 17:36:

@bongobongo https://github.com/bongobongo like i already said, i want to create a chrome extension which does not support web workers. i just tried it :(

@danieldaeschle https://github.com/danieldaeschle , have you looked into this: https://bugs.chromium.org/p/chromium/issues/detail?id=357664#c3

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/infinitered/nsfwjs/issues/94#issuecomment-488001585, or mute the thread https://github.com/notifications/unsubscribe-auth/AHKTWHAKXDBRIT67JTJT5JLPTBRQNANCNFSM4HHYUUNA .

[bongobongo]

@danieldaeschle , do you know that you have to use OffscreenCanvas to use the NSFWJS module in a Web Worker? Like this - place it first in the Web Worker JavaScript file:

`if (typeof OffscreenCanvas !== 'undefined') { // currently tested and works in my Chrome and Opera

self.document = {
    createElement: () => {
        return new OffscreenCanvas(640, 480);
    }
};
self.window = self;
self.screen = {
    width: 640,
    height: 480
};
self.HTMLVideoElement = function() {};
self.HTMLImageElement = function() {};
self.HTMLCanvasElement = OffscreenCanvas;

}`

[danieldaeschle]

Thank you, i'll try this.

bongobongo [email protected] schrieb am Di., 30. Apr. 2019, 17:46:

@danieldaeschle https://github.com/danieldaeschle , do you know that you have to use OffscreenCanvas to use the NSFWJS module in a Web Worker? Like this - place it first in the Web Worker JavaScript file:

`if (typeof OffscreenCanvas !== 'undefined') { // currently tested and works in my Chrome and Opera

self.document = { createElement: () => { return new OffscreenCanvas(640, 480); } }; self.window = self; self.screen = { width: 640, height: 480 }; self.HTMLVideoElement = function() {}; self.HTMLImageElement = function() {}; self.HTMLCanvasElement = OffscreenCanvas;

}`

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/infinitered/nsfwjs/issues/94#issuecomment-488005415, or mute the thread https://github.com/notifications/unsubscribe-auth/AHKTWHCTQVYJE3R2BGOUHPLPTBSWDANCNFSM4HHYUUNA .

[danieldaeschle]

I tried it with cpu instead of gpu which does not require a canvas. That also didn't work.

Daniel Däschle [email protected] schrieb am Di., 30. Apr. 2019, 17:48:

Thank you, i'll try this.

bongobongo [email protected] schrieb am Di., 30. Apr. 2019, 17:46:

@danieldaeschle https://github.com/danieldaeschle , do you know that you have to use OffscreenCanvas to use the NSFWJS module in a Web Worker? Like this - place it first in the Web Worker JavaScript file:

`if (typeof OffscreenCanvas !== 'undefined') { // currently tested and works in my Chrome and Opera

self.document = { createElement: () => { return new OffscreenCanvas(640, 480); } }; self.window = self; self.screen = { width: 640, height: 480 }; self.HTMLVideoElement = function() {}; self.HTMLImageElement = function() {}; self.HTMLCanvasElement = OffscreenCanvas;

}`

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/infinitered/nsfwjs/issues/94#issuecomment-488005415, or mute the thread https://github.com/notifications/unsubscribe-auth/AHKTWHCTQVYJE3R2BGOUHPLPTBSWDANCNFSM4HHYUUNA .

[bumpyflea]

@danieldaeschle - did u have any progress on making the chrome extension? I have just started looking building one myself, and then saw ur comment above saying u were trying it.

[GantMan]

I'd be interested in seeing how this is performing on the latest TensorFlow.js

I know they've been working on the blocking UI problem.

Additionally, I've really shrunk the model, so once it's loaded into memory, it should be super quick!!!1

[danieldaeschle]

Is there any progress in solving the blocking UI issue?

[GantMan]

I've found using the small model doesn't even appear to cause any complication. I haven't tried the larger model in a while.

[danieldaeschle]

Which small model? How long does it take to initialize?

[GantMan]

By default https://nsfwjs.com/ uses the small model. You can change it to the larger model in the dropdown. I have a new small-medium sized model that is more accurate, as well.