Deduplicate model download/registry functionality

[kpu]

While I generally don't mind what happens in python bindings, @jerinphilip shouldn't be doing a python-only model registry. Then the go people get angry when there isn't an equivalent in go or whatever. Language bindings should be wrappers around shared functionality with some language specific sugar on top. There is C++ code in translatelocally with a model registry. We should decide what repo it belongs in and only have one instance of this.

[jerinphilip]

I tried to propose consolidation of inventory previously but was met with pushback (https://github.com/browsermt/bergamot-translator/issues/229#issuecomment-939475419). I do not particularly disagree with what's mentioned in the previous exchange regarding the same, translateLocally is doing inventory well and I appreciate how the software is evolving over experimentation and time. For iBus engine, I do find a separated inventory useful - in which case I can piggyback on translateLocally itself.

There is also the question of bringing additional dependencies into this repository that also feeds places that seem to aggressively optimize to minimize dependencies.

[jelmervdl]

It sounds quite complex to me to implement all of that in cpp in such a way that it still plays nice with whatever bindings you interface with. It would entail we would need at least dependencies for tar, gzip, ssl, http, json parsing, checksumming in bergamot-translator. In TranslateLocally it was easy to do because Qt gives us almost all of that already.

Other considerations are the differences & conventions for each of these platforms. For example, how they treat adding a self-signed root certificate you want to use to connect to your own model repository? Integrating the download into the event loop so you can have progress updates or cancel a slow/stuck download? Having some control over where the models are stored so you can write proper Windows and Apple desktop apps? Supporting all of that also would gives bergamot-translator quite an increase in API surface.

And for low-level bindings like WASM it would not work at all. As far as I know, you can't even get a normal TCP socket in there.