buf icon indicating copy to clipboard operation
buf copied to clipboard

[Feature Request] allowed treat local module as remote deps

Open wenerme opened this issue 2 years ago • 3 comments

In a mono repo, protos layout like

  • buf.work.yaml
  • buf.gen.yaml
  • protos
    • bundle
      • googleapis
        • buf.yaml
    • service-a
      • buf.yaml
    • service-b
      • buf.yaml

The protos/bundle/googleapis is some proto from googleapis. treat as remote deps I mean, when generate, should not generate this module.

Maybe buf.work.yaml support something like deps, or allowed buf.gen.yaml to exclude the module ?

I want to use BSR too, but BSR can not be accessed from everywhere.

Workaround

buf generate <module>

but when there are a lot modules, this become cumbersome to maintain.

wenerme avatar Mar 27 '22 21:03 wenerme

Based on your tree above, I assume you have a buf.work.yaml with the following:

version: v1
directories:
  - protos/bundle/googleapis
  - protos/service-a
  - protos/service-b

Like you've already described in your workaround, if you want to generate for your files without the bundle/googleapis files you need to run buf generate multiple times, one for each module you want to generate for. For example, from the root of your tree:

$ buf generate protos/service-a
$ buf generate protos/service-b

We don't have any plans to extend the buf.work.yaml to recognize some directories as remote dependencies like you suggest. The buf.work.yaml only concerns itself with local modules (see the docs for more information).


With that said, why do you even need the bundle/googleapis directory locally at all? Unless you're making edits to the module itself, you should just be able to use the remote dependency. I wonder if you could better solve this by removing bundle/googleapis altogether, then just running buf generate on the buf.work.yaml so that it delegates to each of the modules defined in it (service-a and service-b in this case).

amckinney avatar Mar 29 '22 01:03 amckinney

I want to use remote deps, but it's a sad story in China, have no choice.

wenerme avatar Mar 29 '22 03:03 wenerme

@amckinney I've got a case in ArgoCD repo, where there is a proto file and code which are used as a dependency for other protos and should not be modified by buf. These files are generated by external kubernetes binaries and I see no way how to itegrate them into buf. Is it possible to use it as a dependency without uploading it to registry or at least add some config parameter where it can be used as a source but excluded from generation process?

AndrewChubatiuk avatar May 04 '22 06:05 AndrewChubatiuk

Closing this for now - we don't have plans to extend the functionality of buf.work.yaml anytime soon, apologies.

bufdev avatar Jan 14 '23 19:01 bufdev