node-libzim icon indicating copy to clipboard operation
node-libzim copied to clipboard

Published 20 hours ago • verified publisher icon openzim

Batch 'addArticle's

Open ISNIT0 opened this issue 5 years ago • 4 comments

The majority of CPU time in CPP (18.8%) when using node-libzim seems to be calls to v8::internal::Builtin_HandleApiCall.

I'm not sure how much it would improve the performance, but we could try creating an addArticles method in CPP and automatically batch the adding on the JS side. This would keep the same simple API, but potentially improve performance

ISNIT0 avatar May 21 '19 16:05 ISNIT0

@ISNIT0 Have a look to https://github.com/nodejs/node/issues/17996, might be completely insignificant.

kelson42 avatar May 22 '19 09:05 kelson42

@kelson42 in that issue, the total percentage was 0.1%, in our case it's 18.8%, so I think it matters much more for us 😀

ISNIT0 avatar May 22 '19 10:05 ISNIT0

@audiodude Do you have some kind of insight on this. Actually I had forgotten about this issue, and that might be a good way to speed-up MWoffliner.

kelson42 avatar Jun 24 '24 08:06 kelson42

I'm not sure. I think mwoffliner will continue to be I/O bound because of all the network calls.

audiodude avatar Jun 24 '24 14:06 audiodude