penumbra icon indicating copy to clipboard operation
penumbra copied to clipboard

Add protos for TCT crate structs

Open grod220 opened this issue 2 years ago • 2 comments

The TCT crate has a few important structs that do not have protobuf equivalents. The one the web repo uses the most is Updates and all of its fields: https://github.com/penumbra-zone/penumbra/blob/6897901b37f5c72d4b5c593167f0e0b933dd2eb5/crates/crypto/tct/src/storage.rs#L279-L290

Without protobuf equivalents, we don't have types in the web repo for these entities. So we type them ourselves w/ zod. It is a runtime validation and not fully extensive (kinda our best effort). Also, it will not stay in sync if things change with those structs.

Proposed solution

Add protos and attributes for these (likely needs the impls too?)

#[serde(into = "pb::TctUpdates", try_from = "pb::TctUpdates")]
pub struct Updates {
...

This way, our bufbuild tooling will automatically spit out the Typescript types that other codebases can use for their block processing.

grod220 avatar Mar 27 '24 17:03 grod220

Do these need to all have protos? So far, we haven't defined a stable data interchange format for the TCT internals on purpose.

What's the benefit of having runtime type validation for the TCT internals? Aren't they going to be produced by the same code that's consuming them?

hdevalence avatar Mar 28 '24 03:03 hdevalence

Do these need to all have protos?

Ideally, there would be some way that non-rust systems consuming these can have native types that represent them and are packaged + versioned. In the past, the bufbuild proto type generation seems to be the canonical route.

What's the benefit of having runtime type validation for the TCT internals?

  • Runtime validation
    • This is more related to web repo specifics. Given TCT does not have types, we have to manually type them all. We could write them in typescript, but we do not get any indication on a breaking change (aka TCT struct field name change). We'll just get a weird error that we have to manually trace. If we type them with zod, we'll receive runtime validation that specifically tells us if the types end up breaking (and where).
  • TCT internals
    • I think my main point is that these types are not internal. They are used within our block processor and a number of rpc methods.

Aren't they going to be produced by the same code that's consuming them?

Yes, at the moment, every non-rust external consumer of TCT types has to manually type them.

grod220 avatar Mar 28 '24 09:03 grod220