cdk-rs
cdk-rs copied to clipboard
dfinity
feat: allow specifying custom serializer / deserializer in macros
Description
Currently, when using the macros exposed by this cdk (query, update, etc.), the request and response must be serialized and deserialized using candid.
This PR allows developers to pass in custom functions to handle serialization and deserialization.
We have been using the MessagePack serializer within OpenChat for many months now using the change in this PR.
How Has This Been Tested?
We have been using this within OpenChat for many months now and have had no issues with it.
Checklist:
- [x] The title of this PR complies with Conventional Commits.
- [x] I have edited the CHANGELOG accordingly.
- [ ] I have made corresponding changes to the documentation.
When there is only a single value being (de)serialized, candid forces you to use a 1 item tuple containing that value.
I didn't follow that same pattern here, when there is only a single value being (de)serialized, the custom serializer function serializes that value directly rather than that value wrapped in a 1 item tuple, similarly, the deserializer function must return that value directly rather than that value wrapped in a 1 item tuple.
I think this feels more natural but we can change it to match candid if need be.
What's the motivation for using a different wire format? This fragments the ecosystem, and is not compatible with the rest of the SDK tooling. You are free to fork the CDK and build tooling around that, but I don't think the official Rust CDK should support this.
Candid is still the default, this change simply makes it possible to use other formats.
In OpenChat we switched away from candid so that we could take advantage of the serde attributes and it has made our progress far quicker. When using candid we couldn't extend enums or add non-optional fields to structs, now that is trivial.
If the goal of the IC is to have all of the world's services running on it then it should allow developers to pick any serializer they want.
Could this be used as convenience over arg_data_raw and reply_raw for skipping unnecessary serialization/deserialization in canisters that proxy calls to other canisters?
When using candid we couldn't extend enums or add non-optional fields to structs, now that is trivial.
We are working on the fix to support extensible variant. Adding non-optional fields is always considered a breaking change, not matter which wire format you use. For stable variable upgrade, you are free to use format other than Candid. But at the interface level, we think the backward compatibility check is the key to enable open services that can communicate with each other.
If the goal of the IC is to have all of the world's services running on it then it should allow developers to pick any serializer they want.
The goal is to enable open services that can depend on each other. If two canisters are using different wire formats at the interface level, how do they call each other? Not to mention all the toolings around canisters. I don't care which wire format we use, but for interoperability, we have to stick with one format.
When using candid we couldn't extend enums or add non-optional fields to structs, now that is trivial.
We are working on the fix to support extensible variant. Adding non-optional fields is always considered a breaking change, not matter which wire format you use. For stable variable upgrade, you are free to use format other than Candid. But at the interface level, we think the backward compatibility check is the key to enable open services that can communicate with each other.
If the goal of the IC is to have all of the world's services running on it then it should allow developers to pick any serializer they want.
The goal is to enable open services that can depend on each other. If two canisters are using different wire formats at the interface level, how do they call each other? Not to mention all the toolings around canisters. I don't care which wire format we use, but for interoperability, we have to stick with one format.
By using serde(default = "...") you can define a non-optional field within your Rust type, while allowing it to be optional to callers of your API. We use this a lot when upgrading our OpenChat user canisters, since while the upgrade is in progress canisters of the old version may call into canisters of the new version, we then remove the attribute once everything is up to date.
Within OpenChat we use MessagePack for all internal canister to canister communication, and our public APIs are all candid.
Many services won't care about integrating with other services and will simply want to use whichever format they are most comfortable with regardless of what the rest of the ecosystem is using.
I think its fine for Dfinity to recommend that people use candid, but developers should still have the option of using whatever they want to.
Could this be used as convenience over
arg_data_rawandreply_rawfor skipping unnecessary serialization/deserialization in canisters that proxy calls to other canisters?
Yup the (de)serializer could simply do nothing to the bytes and pass them through.