build: Use `buf` to compile Protobufs

[judahrand]

What this PR does / why we need it:

I stumbled across buf (https://buf.build) which looks like a really useful tool for making compiling Protobufs much easier. Thought I'd try integrating it into the Feast build process.

Which issue(s) this PR fixes:

Fixes #

[feast-ci-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: judahrand To complete the pull request process, please assign woop after the PR has been reviewed. You can assign the PR to them by writing /assign @woop in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[achals]

Thanks for the PR @judahrand ! Funny timing, I also stumbled on buf and thought it looked cool, so thanks for trying to integrate it here!

[judahrand]

Thanks for the PR @judahrand ! Funny timing, I also stumbled on buf and thought it looked cool, so thanks for trying to integrate it here!

The remote Protobuf compilation is especially handy! Looks like it can remove a ton of boiler plate dependency management / installation.

[codecov-commenter]

Codecov Report

Merging #2711 (4ab8c67) into master (cebf609) will decrease coverage by 21.52%. The diff coverage is 0.00%.

@@             Coverage Diff             @@
##           master    #2711       +/-   ##
===========================================
- Coverage   80.10%   58.57%   -21.53%     
===========================================
  Files         167      167               
  Lines       13976    13952       -24     
===========================================
- Hits        11195     8173     -3022     
- Misses       2781     5779     +2998     

Flag	Coverage Δ
integrationtests	?
unittests	58.57% <0.00%> (+0.09%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
setup.py	0.00% <0.00%> (ø)
.../integration/online_store/test_online_retrieval.py	16.84% <0.00%> (-83.16%)	:arrow_down:
sdk/python/tests/utils/online_read_write_test.py	19.35% <0.00%> (-80.65%)	:arrow_down:
...fline_store/test_universal_historical_retrieval.py	24.17% <0.00%> (-75.83%)	:arrow_down:
.../integration/online_store/test_universal_online.py	17.70% <0.00%> (-72.79%)	:arrow_down:
...ests/integration/e2e/test_python_feature_server.py	28.35% <0.00%> (-71.65%)	:arrow_down:
sdk/python/feast/wait.py	23.52% <0.00%> (-70.59%)	:arrow_down:
...gration/registration/test_feature_service_apply.py	31.25% <0.00%> (-68.75%)	:arrow_down:
sdk/python/feast/infra/online_stores/redis.py	28.39% <0.00%> (-66.67%)	:arrow_down:
sdk/python/tests/integration/e2e/test_usage_e2e.py	33.87% <0.00%> (-66.13%)	:arrow_down:
... and 72 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update cebf609...4ab8c67. Read the comment docs.

[judahrand]

There's still some tidy up but this seems to be loosely working

[pyalex]

Sorry for the dumb question, but what problem is being solved here?

[judahrand]

Sorry for the dumb question, but what problem is being solved here?

No particular specific problem in this project. But it does offload all the complexity of compiling the Protobuf definitions into code on multiple languages to a single tool / config location and does away with some of the logic around protoc plugins. More of this logic can be removed than I have so far I think.

The remote code generation makes having a working build env trivial (no need to have the protoc plugins installed etc.)

The buf tool also has some additional nice functionality such as linting and formatting of the Protobuf files.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.