Tidy up (and document) `.proto` files

[NimJay]

trafficstars

Describe request or inquiry

I have created this issue to request tidy up of (and/or documentation for) these two issues:

1. .proto File Duplication - We might be able to tidy this up.
2. Inconsistent use of .protos - Most — if not, all — of this inconsistency is inevitable (because gRPC code generation works differently for the various programming languages used by the Online Boutique microservices).

1. .proto File Duplication

• The following files are identical:
  • /pb/demo.proto
  • /src/adservice/src/main/proto/demo.proto
  • /src/currencyservice/proto/demo.proto
  • /src/paymentservice/proto/demo.proto
• The following file's contents is a subset of the contents of the files above: /src/protos/Cart.proto
• The follow files are also identical:
  • /pb/grpc/health/v1/health.proto
  • /src/paymentservice/proto/grpc/health/v1/health.proto
  • /src/currencyservice/proto/grpc/health/v1/health.proto

2. Inconsistent Use of .protos

The way in which a microservice uses the .proto files and generates its gRPC code depends on the programming language of that microservice. Ideally, gRPC (and .proto file) usage would be better documented for each microservice. Here's some info about the overall issue, for now:

Microservice	Language	How are .protos used?	Generated gRPC Code	.proto file used
adservice	Java			/src/adservice/src/release/v0.3.6/proto/demo.proto
cartservice	C#			/src/protos/Cart.proto
checkoutservice	Go	See /src/checkoutservice/genproto.sh	/src/checkoutservice/genproto/demo.pb.go	/pb/demo.proto
currencyservice	JavaScript	gRPC code is generated during runtime, so we just need to feed the .proto file(s) into the gRPC library when we run the Node.js server.		/src/currencyservice/proto/demo.proto
emailservice	Python	See /src/emailservice/genproto.sh	/src/emailservice/demo_pb2.py & /src/emailservice/demo_pb2_grpc.py	/pb/demo.proto
frontend	Go	See /src/frontend/genproto.sh	/src/frontend/genproto/demo.pb.go	/pb/demo.proto
loadgenerator	Python	The loadgenerator just makes HTTP requests to the frontend service. No gRPC is used.
recommendationservice	Python	See /src/recommendationservice/genproto.sh	/src/recommendationservice/demo_pb2.py & /src/recommendationservice/demo_pb2_grpc.py	/pb/demo.proto
shippingservice	Go	See /src/shippingservice/genproto.sh	/src/shippingservice/genproto/demo.pb.go	/pb/demo.proto
paymentservice	JavaScript	gRPC code is generated during runtime, so we just need to feed the .proto file(s) into the gRPC library when we run the Node.js server.		/src/paymentservice/proto/demo.proto
productcatalogservice	Go	See /src/productcatalogservice/genproto.sh	/src/productcatalogservice/genproto/demo.pb.go	/pb/demo.proto

What purpose/environment will this feature serve?

• Tidying up the .protos will help reduce the learning curve of this repo, and make it easier for new developers to contribute and use Online Boutique.

What inspired this issue?

• A team at Google is using Online Boutique to showcase the Retail API. The team faced some friction while trying to refactor the .protos for the Python microservices and the frontend service.

[mathieu-benoit]

Could be great to consider this one in the meantime maybe? https://github.com/GoogleCloudPlatform/microservices-demo/issues/1

[mic-max]

In OpenTelemetry we started a fork and have some work done to remove these duplicate .proto files as part of the following PRs:

• https://github.com/open-telemetry/opentelemetry-demo-webstore/pull/118
• https://github.com/open-telemetry/opentelemetry-demo-webstore/pull/117
• https://github.com/open-telemetry/opentelemetry-demo-webstore/pull/116
• https://github.com/open-telemetry/opentelemetry-demo-webstore/pull/108
• https://github.com/open-telemetry/opentelemetry-demo-webstore/pull/105
• https://github.com/open-telemetry/opentelemetry-demo-webstore/pull/92

Each service's Dockerfile copies from /pb/demo.proto and runs protoc to generate the language files at build time rather than checking them in. Just FYI :)

[mathieu-benoit]

Thanks for sharing @mic-max, much appreciated!

[NimJay]

Wow, @mic-max, thank you so much for sharing your work!

I realized that the pull-requests (PRs) you shared contain changes (e.g., changes specific to the timestamp.proto that's not found in our* repo) that are outside the scope of this GitHub issue (#814), so we might have to do some careful sifting to leverage the PRs. But we'll definitely take inspiration from them (and I'm sure we'll find them useful). And, of course, feel free to create PRs into our* repo (containing just the .proto file relocations).

"our* repo" = github.com/GoogleCloudPlatform/microservices-demo

[NimJay]

Commenting to cool this issue down (from our team's out-of-SLO list): We should still address this issue. I'll leave it open.

[bourgeoisor]

Cooling down SLO-- Still in backlog.

[NimJay]

Team out-of-SLO cooldown: no progress on this issue. This work would be useful — I'll leave this open.