Add support for name filtering in `buf generate` command

[remyleone]

If I have multiple generators defined in my buf.gen.yaml, sometimes I want only to launch a subset of the generators. Could it be possible for the buf generate command to accept a name option/filter that will only select a subset of the generators defined in the config file or pass through the template option?

[robbertvanginkel]

We currently treat buf.gen.yaml as all or nothing in terms of running all defined codegenerators, similar to how templates are defined and work with remote generation on the BSR.

If you want to be able to run different sets of generators, we currently recommend that you keep multiple buf.subset.gen.yaml files and use the --template flag when doing buf generate. Note that the --template doesn't only take files but can also take a json encoded template from the commandline. Is that a solution that would work for you?

[remyleone]

That would mean duplicating configuration for all generators in all the subsets files :/ the number of files to keep on disk will keep growing with this solution. Filtering seems to be more elegant.

[amckinney]

I want to challenge this a bit. I'm not sure that you'd need to duplicate anything across any files - you just need to write your files granularly so that they can composed together by separate buf generate invocations. I think the number of files you'd need is equivalent to the number of buf generate --name invocations you'd need to write if we supported a --name filter, right?

Regardless, a --name filter is a reasonable request - we can consider it, but we have taken a strict stance that input configuration goes in flags, template configuration goes in buf.gen.yaml, and this would make that split murkier.

I also worry about the implication on other features in the buf.gen.yaml like Managed Mode. If we have a way to select individual plugins, would we also need a way to enable/disable different parts of the managed key? This could quickly get unwieldy if so.

[anandwana001]

We currently treat buf.gen.yaml as all or nothing in terms of running all defined codegenerators, similar to how templates are defined and work with remote generation on the BSR.

If you want to be able to run different sets of generators, we currently recommend that you keep multiple buf.subset.gen.yaml files and use the --template flag when doing buf generate. Note that the --template doesn't only take files but can also take a json encoded template from the commandline. Is that a solution that would work for you?

@robbertvanginkel How to keep multiple buf.subset.gen.yaml files? It rises the duplicate name issue in project.

If we prefer a subset like this in generating command only, then how does it affect buf.gen.yaml file? Does it override or not? https://docs.buf.build/generate/usage#common-use-cases

[bufdev]

Here's how we maintain multiple sets: https://github.com/bufbuild/buf/tree/main/data/template

You can use the --template flag https://github.com/bufbuild/buf/blob/main/make/buf/all.mk#L104

[bufdev]

I think for now we're going to close this: the canonical way to do this currently is with having multiple buf.gen.yaml files, and we don't want to mix responsibilities between flags and config files as described above. We can revisit in the future if there's a big need for this.

[bufdev]

We're going to reconsider this

[AndreasBergmeier6176]

Our use case would be to filter the files that get considered in a plugin entry. E.g. when we have foo.proto and bar.proto it would be great to be able to directly specify, which protos will be used for the plugin invocation.

Basically a mechanism which loads the path option from buf.gen.yaml.