feat(entproto): support for named proto messages

[pavelsmejkal]

Hello, I would like to get your feedback on my proposal to add support for more granular control over proto message generated by the entproto package. When accepted I can add tests and missing things like support for edges, etc...

The motivation here is to have full control over proto messages that are being generated based on ent schema. The contrib package is great but GRPC service generated out of it doesn't yet provide enough extensibility.

When accepted. I would like to explore as well the extensibility of the GRPC service. How to make customizable. Or reusable in user's implementation.

The proposal contains several features:

• Ability to specify a different names for fields to decouple from the DB layer. For example for a sake of compatibility with existing systems.
• Named messages, a concept that allows creating multiple custom proto messages based on fields, edges, or fields groups,
• Support for extra fields that will not have a counterpart in the database. To allow model more complex scenarios. JSONB etc.
• Support for "field groups" that allow you to work with fields more efficiently in named messages.
• Allow to not render the ID field

The go schema definition

type Car struct {
	ent.Schema
}

func (c Car) Annotations() []schema.Annotation {
	return []schema.Annotation{
		entproto.Message(
			entproto.NamedMessages(
				entproto.NamedMessage("CarUpdate").
					WithGroups(
						c.Groups().ByName("common", "dates"),
					).
					WithExtraFields(
						entproto.ExtraField("extra_1", 4).WithType(entproto.TypeBool),
						entproto.ExtraField("extra_2", 5).WithType(entproto.TypeString),
					).
					WithSkipID(true),
				entproto.NamedMessage("CarUpdateDates").WithGroups(
					c.Groups().ByName("dates"),
				).WithSkipID(true),
			),
		),
	}
}

func (Car) Groups() *entproto.FieldGroups {
	return entproto.Groups().
		Group("common", func(fg *entproto.FieldGroup) {
			fg.Fields = []ent.Field{
				field.String("model").Annotations(entproto.Field(2)),
			}
		}).
		Group("dates", func(fg *entproto.FieldGroup) {
			fg.Fields = []ent.Field{
				field.Time("registered_at").Annotations(entproto.Field(3, entproto.FieldName("RegisteredAt"))),
			}
		})
}

// Fields of the Car.
func (c Car) Fields() []ent.Field {
	return c.Groups().Fields() // ByName("common", "dates").Fields()
}

Generated proto

// Code generated by entproto. DO NOT EDIT.
syntax = "proto3";

package entpb;

import "google/protobuf/timestamp.proto";

option go_package = "outreach.io/enttest/ent/proto/entpb";

message Car {
  int32 id = 1;

  string model = 2;

  google.protobuf.Timestamp RegisteredAt = 3;
}

message CarUpdate {
  string model = 2;

  google.protobuf.Timestamp RegisteredAt = 3;

  bool extra_1 = 4;

  string extra_2 = 5;
}

message CarUpdateDates {
  google.protobuf.Timestamp RegisteredAt = 3;
}

[rotemtam]

Hey @pavelsmejkal !

Thanks so much for taking the time to think deeply about how to improve the gRPC extension. Indeed it is still early days for the plugin and there is much to be desired.

In general, our approach with Ent is to make small incremental improvements with each PR so code is both easy to review and results in better more modular design in general. So I think it would be good to consider each improvement in separate and work on different small features.

Ability to specify a different names for fields to decouple from the DB layer. For example for a sake of compatibility with existing systems. I believe this can already be done with StorageKey?

Named messages, a concept that allows creating multiple custom proto messages based on fields, edges, or fields groups, Support for "field groups" that allow you to work with fields more efficiently in named messages.

I think I understand the use-case for this one, but I have an issue with creating another mini DSL for defining proto in Ent. The appeal of extensions such as entproto is that it uses the existing schema modeling capabilities of Ent. So two directions for design would be, how do we import existing proto definitions to be used in tandem with an Ent schema, or how we take Ent's existing facilities (JSON/Other fields) and use them for this use case. Perhaps we can revive the discussion at https://github.com/ent/ent/discussions/1690 for this?

Allow to not render the ID field Not sure I understand this one, can you elaborate on the use case and how this would modify the generated code?

Thanks!

[pavelsmejkal]

Thank you for a feedback and the direction, that was very much what I was looking for. Will explore more the capabilities and will bring the proposal to ent/ent#1690. I have removed my original post since I would like to think about it more.

Edit: This PR was a showcase then PR, i was exploring ways how to work with protobufer and it was more like a POC.

[rotemtam]

Sure thing. You're more than welcome to join our discord server: https://discord.gg/qZmPgTE6RX

[waltcow]

This feature is awesome to me

[rverma-dev]

Also this need to be supported in ent-proto too for protocol to ent generation