protoc-gen-doc files are overwritten in nested proto directory scenarios

[ganeshs]

I have a module with two proto files in a directory. File1 is under directory dir1 and File2 is under dir2. Both dir1 and dir2 are under dir0.

I have the below configuration for the "doc" plugin.

    - name: doc
      flags: markdown,test.md
      output: docs

when I invoke prototool by passing dir0 as input, I see it protoc command twice (once for each proto file). This causes the second run to overwrite the doc generated from the first one. Is there a workaround for this?

[ganeshs]

Looks like the code the assuming that the plugin will take care of creating the generated files under the respective folders. In case of the doc gen plugin, it takes in the name of the file to generate and the output folder for the entire module. So when prototool runs per directory, the out path is not changing and hence getting overwritten.

I'm suggesting an enhancement to the plugin options to include a new boolean flag "source_relative" which will set the out path to a folder relative to the proto file. I can work on this and file a PR for this.

[mattbailey]

This is also true for protoc-gen-swagger

[bvisness]

This seems to be broken more generally in a way that a source_relative flag won't fix.

The general way that protoc plugins work is that they are given a CodeGeneratorRequest, which contains a list of files to compile and some extra parameters (like the data passed in by flags). Plugins like protoc-gen-doc and protoc-gen-swagger are designed to be used in a single invocation of protoc that compiles all your proto files at once - that way, it has the context it needs to generate links between all the types in your proto definitions.

Prototool seems to break this by doing one invocation of protoc per plugin and directory. When invoked this way, each individual run only sees one directory at a time and is missing lots of necessary context. It then also just dumps its output into the same place as every prior run, overwriting any runs that came before.

So, Prototool only works with plugins that produce one output file per proto file, and plugins that only output a single file for the whole run are broken.

This is very frustrating and I don't understand the rationale behind it. It might make sense if you could specify different config per directory, but I don't see anything to that effect, and I don't see anything that would prevent Prototool from just doing one big protoc run.