Feature: Refactor client model download into a separate module

[mncharity]

Some time ago on linux, I found using a single webcam simultaneously from multiple processes added at least a frame of latency, which was problematic for VR/AR with slow webcams. If one is running other analyses in addition to posenet, that incentivizes combining them all into a single custom server, rather than using multiple turnkey servers. One can then also make application-specific performance tradeoffs and optimizations (like running posenet on only a part of the frame).

Thus I'm adding posenet to my existing server. But I'd still like it to work offline. So ideally, I'd like posenet-for-installations's model downloading functionality available as a separate module. Say a module which runs client/downloadModels.js upon install, and has client/src/models.ts as its entry point (but with BASE_URL refactored as an argument to loadModel). One user's story, FWIW.

Thank you for your work - it was quite helpful.

[oveddan]

Yes a standalone thing would be great. A lot of people would find it useful to just have an offline version of PoseNet. I believe this could be just part of the original tfjs-models repo, with the first step being allowing posenet to be loaded from a custom url: https://github.com/tensorflow/tfjs/issues/616

[oveddan]

@mncharity how would you imagine the integration of that standalone feature working in your project?

[mncharity]

My quick-and-dirty approach was: a standalone module with a script "postinstall": "node downloadModels.js"; a project module build step which uses path.dirname(require.resolve('tfjs-models-weights-posenet-files'))+"/data" to find the download directory, and then links to it under the project's public/; and an import loadModel from tfjs-models-weights-posenet-files with a url argument added to loadModel(thelinkedurl, multiplier).

I was getting Error: socket hang up install failures until I added a large {timeout:10000} to https.get in downloadModels.js. Perhaps the download parallelism might be interacting poorly with yarn build system parallelism, creating problematic latencies? If so, it might be worth reducing the download parallelism?

I wondered whether posenet.CheckpointLoader accepts file: URLs, so this approach could be used with electron apps which aren't running a web server. But I haven't checked yet.

Instead of linking to the download directory, it might be more portable and robust to copy it.