go-ipld-prime icon indicating copy to clipboard operation
go-ipld-prime copied to clipboard

cidlink.Link in ipld.Link never being a pointer requires allocs

Open mvdan opened this issue 3 years ago • 1 comments

Interfaces are always a (type, pointer) pair, so if one shoves a non-pointer value in them, that's always an alloc. This is the case with cidlink.Link, as it's a struct.

It would be theoretically possible for the compiler to support "pointer-sized" interface values without separate allocs and pointers, but even that's impossible here, as the type ends up being struct{ struct{ string } }, which is twice the size of a pointer.

Why does this matter? For example, when converting back and forth between merkledag.ProtoNode and dagpb.PBNode, we iterate over the inner links to convert them too. Because of the type above, each link converted is an extra alloc for the interface conversion. And I can't easily batch those allocations, because the allocation is part of the runtime's "convert type to interface" logic and I can't control it, as far as I can see. If I convert a *cidlink.Link to ipld.Link it works, but then other packages break when they expect a non-pointer inner type.

The backstory here is https://github.com/ipld/go-ipld-prime/pull/96. It seems to me like "want interface" plus "want easy key-ability" plus "want no allocs" is a near impossible mix to get all at once.

mvdan avatar Mar 28 '21 10:03 mvdan

Personally, I see two options:

  1. Force a pointer. Makes map keys and other comparisons harder, but this might be okay if it's always go-ipld-prime who creates the cidlink.Link pointers. Another option would be to provide an API that takes a cidlink.Link and returns a unique *cidlink.Link for it, though one would have to be careful to not have an ever-growing global map.

  2. Remove the interface, and use cidlink.Link directly instead of link.Link. I think this is the better option in the long run, because the ability to have multiple link implementations seems secondary once the main implementation is used everywhere and is efficient. This might depend on whether we want to stick with go-cid.

mvdan avatar Mar 28 '21 10:03 mvdan