Error while using `grpc-tools.protoc` for `protoc` subcommand

[98sean98]

Console error:

  File "/Users/seanchok/Documents/rpc/.venv/lib/python3.10/site-packages/protoletariat/fdsetgen.py", line 137, in generate_file_descriptor_set_bytes
    subprocess.check_output(
  File "/Users/seanchok/.pyenv/versions/3.10.0/lib/python3.10/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/Users/seanchok/.pyenv/versions/3.10.0/lib/python3.10/subprocess.py", line 501, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/Users/seanchok/.pyenv/versions/3.10.0/lib/python3.10/subprocess.py", line 966, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/seanchok/.pyenv/versions/3.10.0/lib/python3.10/subprocess.py", line 1842, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'python -m grpc_tools.protoc'

Steps to reproduce:

1. install grpc-tools through pip to use protoc

2. generate python bindings

python -m grpc-tools.protoc \
  --proto_path=. \
  --python_out=out \
  --grpc_python_out=out \
  *.proto

3. Use protol protoc

protol \
  --create-package \
  --in-place \
  --python_out out
  protoc \
  --protoc-path "python -m grpc_tools.protoc" \
  --proto-path .
  *.proto

My own research:

1. I read through the proloteriat source code, and suspected that the subprocess.check_output call may not be able to call python to run executable modules just like that. But I'm not entirely sure because the error thrown appears to be FileNotFound.

2. I created a workaround by first generating the FileDescriptorSet file, and then running protol raw instead of protol protoc.

touch descriptor.temp

# generate FileDescriptorSet
python -m grpc_tools.protoc \
  --include_imports \
  --descriptor_set_out=descriptor.temp \
  --proto_path=. \
  *.proto

# run post-processing to change absolute imports generated by protoc to relative imports
protol \
  --create-package \
  --in-place \
  --python-out out \
  raw \
  descriptor.temp

I got the inspiration for the workaround after reading this section of code.

I'm not sure if protol needs to be updated to make running python -m grpc_tools.protoc possible. Or if this workaround should be made known to other people who are using the grpc-tools package as I am.

[cpcloud]

@98sean98 Thanks for opening an issue. I think the workaround is the right solution here. While there aren't a huge number of ways to generate the FileDescriptorSet binary there are enough that I think adding a new subcommand for each one is unnecessary when raw suffices. I'm happy to add usage of grpc_tools.protoc to README.md.

[98sean98]

I see. Would you be happy with me filing a pull request to write a section in README.md for grpc_tools.protoc?

[cpcloud]

I see. Would you be happy with me filing a pull request to write a section in README.md for graph_tools.protoc?

That'd be great!