Aleksei Gorb

Process is going there https://docs.google.com/spreadsheets/d/1oNX9XAILHemks4BW9mXOLb4gnpn7w-j_VBMXwastNY0/edit#gid=0

here are same intentions as in [option 3](https://github.com/openzim/mwoffliner/issues/1100#issuecomment-625750539), just a bit more generalized approach

@kelson42 I'm chiming with the approach with reading from Redis serially then process the chunks in parallel.

not sure why codecov doesn't count jest coverage (`Dowloader.ts / getJsonCb()` for example)

> At least 20% slower with bm.wikipedia.org (with S3 cache) @kelson42 Just got some benchmarks for bm.wiki. Running with: ``` /home/mid/.nvm/versions/node/v12.9.0/bin/node -r ts-node/register /home/mid/_projects/kiwix/mwoffliner/src/cli.ts [email protected] --mwUrl=http://bm.wikipedia.org/ --format=nopic:nopic ``` On localhost:...

> Get stuck with mi.wikipedia.org around 50% of the time (with S3 cache) @kelson42 Run on `master` branch (bfd2f36): 767s - [mi-1-master.log](https://github.com/openzim/mwoffliner/files/5133086/mi-1-master.log) 711s - [mi-2-master.log](https://github.com/openzim/mwoffliner/files/5133087/mi-2-master.log) 703s - [mi-3-master.log](https://github.com/openzim/mwoffliner/files/5133089/mi-3-master.log) Run on...

> My own runs with: Wondering why your timers looks different than mine... anyways we cannot consider `redis-iterate` 20% slower than `master` (or even slower at all) :) Most interesting...

as a quick option for maintainers: it would be good to just include typescript sources in the package