Prepare for Cloud offering.

[cchudant]

Description

Lots of things.

• JWT validation. When enabled (jwt_sign.pem file is present – ecdsa pubkey), it will force the limit of models per user to 1
• Removed cbor on the wire, refactor inputs and outputs into message TensorData struct, as well as tensor facts for RunModel and SendModel
• Make optimize optional. Not in protobuf yet, but implemented server side. Can be used in startup models (from config.toml file)
• Make loading models from config.toml file possible
• Make loading huge models from config.toml file possible
• Model can be sealed, defaults to false.
• "send_model = false" in config.toml
• Tensors may have names. Not yet used in blindai client, but it's here on the wire.
• Max model store (number) for blindai as a service. Defaults to 0, which does not change anything. If set, older models will be removed.
• Update tract, because it was needed for big model support from filesystem.
• Minor changes like adding a lot of context to existing errors (anyhow .context("xx"))
• Add DatumTypes U8, I8, U16, I16, Bool
• Added support for pytorch / numpy tensors directly on RunModel (in client!)
• Client: Support using Mithril Cloud Services
• Config entry: send_model_requires_auth (bool)
• Config entry: allow_sendmodel (bool)
• Config entry: max_model_store (int). When set, models that are pushed beyond this limit will remove old models.

Left to do

• Edit the tests, make them pass, write more.
• Documentation for each of these features.
• Update examples.
• Update readme.

Related Issue

None

Type of change

Warning: The changes to the python API are not backward compatible.

• [x] This change requires a documentation update
• [x] This change affects the client
• [x] This change affects the server
• [x] This change affects the API
• [x] This change only concerns the documentation

How Has This Been Tested?

Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc.

Checklist:

• [ ] My code follows the style guidelines of this project
• [ ] I have performed a self-review of my code
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] My changes generate no new warnings
• [ ] I have updated the documentation according to my changes