New design `__variants__` and `__typeof__`

[1st1]

We'd like to change the current implementation of __variants__ and __typeof__.

__variants__ -> __shape__

__variants__ is long to type and is also an unknown term to both Python and EdgeQL. But many of the use cases of custom-defined types will be creating a type that can satisfy a custom shape in a .select() call. __shape__ covers that and is shorter, so a win-win.

The new variants tree

We brainstormed what different kinds of classes the user might need and discovered that it's a long list. A class with all fields but no id, a class with all fields made optional but with an id, etc. etc.

The only reasonable way out where we have things that are easy to compose and familiarize yourself with is to rely on composition (and not on having tens of classes with long names).

Base classes that manage the "id" field

• class Base -- a base class that has __type__.

  • It's the root of the hierarchy.
  • Actual usable models must to be inherited from NoId, ReqiredId, or OptionalId or it will not be possible to instantiate them in any way.

• class NoId(Base) -- a base class that has no id field in a sense of:

  • id can't be passed to __init__() or to model_validate()
  • id isn't going to be listed in JSON schema or returned in model_dump().
  • save() can assign the object an id, so once saved the object's .id attribute would return the id.
  • after save(), the id field will be picked up and used by __eq__.

• class RequiredId(Base) -- a base class that has a required id

• class OptionalId(Base) -- a base class that has an optional id

No manual id field declaration

Inherit from one of the base classes. We should not allow this:

class MyUser(User.__shape__.NoId):
    id: uuid.UUID

Base classes for properties

• class RequiredProps(Base) -- a base class that has only required props (optional props are ommitted)
• class PropsAsDeclared(Base) -- a base class that has all props as they were declared in the schema
• class PropsAsOptional(Base) -- a base class that has all props as optional (required props become optional with this class)

Base classes for links

Somewhat analogous to the helpers we'll have for props, but links will point to Base variant of the link type. This means that links will accept no-id, required-id, and optional-id flavors of the link type (and that should be accurately reflected in JSON schema).

• class RequiredLinks(Base) -- same as for props
• class LinksAsDeclared(Base) -- same as for props
• class LinksAsOptional(Base) -- same as for props

Useful pre-defined CRUD primitives

• class Create(NoId, PropsAsDeclared, LinksAsDeclared) -- useful for create endpoints -- accept data, validate, add to the DB.
• class Read(RequiredId, PropsAsDeclared)
• class Update(OptionalId, PropsAsOptional)

New __typeof__ behavior, __fields__

• We change __typeof__ to return the actual type of the field.
• We add __fields__ as an index of the fields.

So for the following class:

class Example(GelModel):
    foo = RequiredProperty[std.int, int]

• Example.__fields__.foo would be RequiredProperty[std.int, int]
• Example.__typeof__.foo would be Wrapper[std.int64, int] -- the exact name / mechanism is to be determined.

This would allow this use case:

class MyMovie(Movie.__shape__.NoId):

    # change the field cardinality to *optional* without
    # restating the type
    title: OptionalProperty[Movie.__typeof__.title]
    
    # inherit the field as it's defined in the reflected
    # schema
    release_date: Movie.__fields__.release_date

Edge cases

• Copying fields via __typeof__ and __fields__ into a plain Pydantic model should be prohibited and we need to raise a nice error as early as we can.
• No manual id field declaration
• We prohibit combining NoId, RequiredId, and OptionalId in the __mro__ -- there can be only one of them in the hierarchy

---

cc @elprans @vpetrovykh @scotttrinh

[scotttrinh]

But many of the use cases of custom-defined types will be creating a type that can satisfy a custom shape in a .select() call.

I'm not actually against this name, but given that the classes that one finds here are primarily mixins (see below) this feels slightly off still. I don't have a good succinct suggestion that is better at expressing this than __shape__ though, so not a blocker at all.

The only reasonable way out where we have things that are easy to compose and familiarize yourself with is to rely on composition (and not on having tens of classes with long names).

I think this is a win in general, so 👍

• class Create(Base, PropsAsDeclared, LinksForCreate) -- useful for create endpoints -- accept data, validate, add to the DB.
• class Read(RequiredId, PropsAsDeclared)
• class Update(OptionalId, PropsAsOptional)

Should Read and Update also extend Base like Create does?

New typeof behavior, fields

This makes sense as far as I can understand it, but maybe the example class can be a more realistic example of generating a custom class for some specific purpose just so I can understand how you'd use __typeof__ vs __fields__?

[1st1]

~How about __mixins__~ -- those classes can be used standalone

[scotttrinh]

How about __mixins__

Actually after sleeping on it, __shape__ seems fine even for mixins. "You compose shapes together to get a more complex shape."