plugin-relay: More generalized global ID codec support.

[cometkim]

Currently the @pothos/plugin-relay is a bit opinionated on the global id format and is difficult to customize.

The type definition has following constraints:

• Type names are stored as strings.
• IDs are serialized as string/number/bigint.
• No extra keys other than typename and id are included.

I have production code that doesn't meet the above constraints.

const typeRegistry = new WeakMap<DocumentSchema, number>();
typeRegistry.set(UserSchema, 1);

export function encodeGlobalIDv2(Schema: DocumentSchema, Id: DocumentId) {
  const CODEC_VERSION = 2;
  const repr = {
    v: CODEC_VERSION,
    t: typeRegistry.get(Schema),
    i: Id.buffer, // ArrayBuffer
  };
  const buf = CBOR.encode(repr);
  return buf.toString('base64url');
}

As you can see, the global ID encodings may not use a string representation internally at all, or may have different metadata keys.

I couldn't find a way to use the built-in types in @pothos/plugin-relay.

It would be nice to improve the interface to make it more generic, allowing the use of any custom global ID codecs.

[hayes]

I think generating the same IDs using the existing APIs should be pretty straight forward, but I am not sure what DocumentSchema and DocumentId are in your current version.

In Pothos the internal representation of IDs needs to be typename + id (as a string, or number) because they are used as cache keys in dataloaders, and changing this would be fairly complicated.

This shouldn't interfere with a completely custom codec though.

You would need to tweak things slightly in both your encode and your resolver:

The resolver would probably return base64(id.buffer), the registry would need to be updated to use typenames as strings in a map (instead of a weak map), and then the encoder would need to parse the id buffer back to bytes before passing into your encoder.

Your original list of restrictions is still accurate, but is in place because of Pothos node loading logic

If you are not using Pothos node loading APIs, you can just use normal id fields and handle if encoding any way you want

For typenames, I'm not sure how Pothos would be able to resolve nodes using arbitrary objects for the typenames (what would DocumentSchema be)

For id serialization and extra data, we need something that can be used as a cache key, but it's trivial to encode other data into that key as a string, and if fields on nodes have a parse option to turn it back into structured.data: https://pothos-graphql.dev/docs/plugins/relay#parsing-node-ids

If you can provide more details on what you are trying to do, I can see what we can add to Pothos, or figure out if there are existing APIs for what you're trying to do.

[cometkim]

Well, I'm migrating an existing Mongoose model to a POJO validated w/ Zod schema. I have some abstractions for it, where DocumentSchema is just a Zod schema representing a MongoDB document with an _id: ObjectId field. Something like:

const ObjectIdSchema = z.union([
  z.instanceof(ObjectId),
  z.instanceof(Uint8Array).transform(id => new ObjectId(id)),
  z.string().transform(ObjectId.createFromHexString),
]);

const UserIdSchema = ObjectIdSchema.brand('UserId');

const UserDocument = z.object({
  _id: UserIdSchema,
  //...
});

My goal is to expose these as Relay interfaces.

type User extends Node {
  id: ID!
}

However, I want a more optimized ID codec that avoids unnecessary serializations. At the same time, to ensure compatibility with several older versions of the global ID codec I already have, the ID itself must include codec version information.

So the ID codec must contain the following information:

1. Codec version
2. Type information compressed by a dictionary
3. A byte representation of the ObjectId

Since my goal is to register this with a single definition per document, it's a bit more complex and will require a custom plugin.

I was curious to see how much code from the existing Relay plugin could be reused without vendoring.

[hayes]

How do you handle decoding, and are you trying to use the relay Query.node and Query.nodes fields?

The limitations on id are not really about the encoding side, that would be easy to support, it's about being able to reliably decode the serialized ID back into original id and resolve to the type that needs to be loaded.

I can look into exposing more types/helpers from the plugin, do you know which pieces you would want to reuse?

I am also open to making I'd encoding more flexible, but I might need some more specific ideas on how you would want this to work in Pothos

[cometkim]

How do you handle decoding, and are you trying to use the relay Query.node and Query.nodes fields?

Yes, it was a bit more complicated than I initially thought. It needed something like triples to search the registered info. Like [typename: "User", type: UserDocument, tag: 1].

Now that I see the loader implementation, understand the relay plugin's limitations, and agree that it's necessary. I wish it at least supported custom representative types rather than string/number/bigint.

I'll share more specific usage and ideas for generalization once I've written the custom plugin.

[hayes]

One option might be to support custom types, and have a cache-key function that can be used to derive the key from the data so the data can be more flexible. There is something similar in the auth plugin