go-hamt-ipld icon indicating copy to clipboard operation
go-hamt-ipld copied to clipboard

feat!: use generics instead of cbg.Deferred

Open rvagg opened this issue 1 year ago • 2 comments

Experimenting with generics to do away with the deferred value decoding step.

The generic must conform to [T HamtValue[T]] where:

type HamtValue[T any] interface {
	Equal(T) bool
	MarshalCBOR(io.Writer) error
	UnmarshalCBOR(io.Reader) error
}
  • * pointers are kind of awkward in this, tho I'm not sure I can do much better without heavier changes (e.g. there's situations where you want to pass nil to indicate deletion)
  • The Equal is a little annoying, but necessary to know whether we're mutating in certain situations
  • FindRaw is gone, doesn't make sense anymore I think.
  • Find has a current use-case where you pass nil as the decoder and that means "I just want to know if it's there"; for certain types T it'll be more expensive to decode the full node with T elements than it was to do the cbg.Deferred cbor token skip job. Maybe.
  • Needs some benchmarking in cases like the lotus actor state migration where I'm suspecting this might be of benefit.
  • Maybe this could be /v4 but we still retain the /v3 variant for some cases.

rvagg avatar Aug 02 '24 11:08 rvagg

Experimentation with this going on over at https://github.com/filecoin-project/go-state-types/pull/298 against a >3M entry HAMT.

rvagg avatar Aug 05 '24 02:08 rvagg

Rebased on #121, which has the restored cbor-gen code in it, so this now depends on https://github.com/whyrusleeping/cbor-gen/pull/103 for cbor-gen generics support.

rvagg avatar Aug 05 '24 12:08 rvagg

Bailing on this for now, as per https://github.com/whyrusleeping/cbor-gen/pull/103#issuecomment-2398852941

rvagg avatar Oct 08 '24 05:10 rvagg