hearth
hearth copied to clipboard
Migrate to CBOR as the de facto serialization format
We've been using JSON, which bloats message size, is inefficient to encode and decode floats and integers, and requires conversion of blobs to base64 strings in order to have reasonable fast serialization. This kinda sucks!
A new message format that alleviates these problems should have the following properties:
- Binary format. Efficient data representation, especially for raw byte arrays.
- Schema-less. Can be made human-readable without the need for an external schema definition. Can be directly imported into a scripting environment as an object type.
- Simple. Easy to work with.
CBOR satisfies all of these goals. No more base64! No more processes bottlenecked by ser/de!
To-do:
- #278
- add ciborium as a workspace dep
- remove
serde_with
from schema and DON'T base64-encode anything - add
ciborium
to runtime and use it to serialize events inPubSub
, deserialize messages inSinkProcess
, and serialize responses inRequestResponseProcess
. - swap out
serde_json
forciborium
in init - swap out
serde_json
forciborium
inrun_wasm
example - remove
serde_json
dep from workspace, schema, ctl, init, wasm, server, fs, and runtime. - swap out
serde_json
forciborium
in guest - add
ciborium
to kindling workspace deps - update all of the kindling crates that use
serde_json
tociborium
- remove
ciborium
from kindling
Then, after #197:
- update
ByteVec
with customDeserialize
andSerialize
implementations that read and write the serde bytes data type directly - rework
JsonAssetLoader
intoCborAssetLoader