Support package prefixing for all managed mode languages

[noahseger]

For each managed language in https://docs.buf.build/generate/managed-mode#languages, Buf should support an optional prefix — similar to the way java_package_prefix and go_package_prefix works. This lets us decouple packaging from protobuf directory organization.

In particular, I have a hard time packaging PHP for reuse because the php_namespace is completely controlled by Buf in managed mode, and in unmanaged mode I'd have to ensure every team properly prefixes their protobuf with the correct php_namespace — verbose and error-prone.

[jozuenoon]

Bumping this issue. In managed mode proto options:

option php_namespace = "xxx";
option php_metadata_namespace = "xxx\\Metadata";

...are completely ignored. Is there any development regarding that? This renders PHP plugin mostly unusable in my scenario.

[doriable]

We recently added except and override keys for ruby and csharp and default for obj-c prefix, and I'm currently reviewing the PHP use-case. I want to make sure we capture the right behavior for PHP users -- would the ideal be something along the lines of:

managed:
  enabled: true
  php_namespace_prefix:
    default: My\\Namespace # this gets prefixed before all namespaces derived from the package name
    except:
      - buf.build/<owner>/<module> # the module name of a dependency that should be exempt from the default and basically any prefixing behavior
    override:
      buf.build/<owner>/<module>: Your\\Namespace # a module that will get Your\\Namespace as the prefix instead 
  php_metadata_suffix:
    default: Metadata # instead of using the default GPBMetadata
    except:
      - buf.build/<owner>/<module> # similar to above, exempt this module
    override:
      buf.build/<owner>/<module>: Foo # a module that will have metadata suffixed with Foo instead

Where you are able to control the prefix for the namespace and suffix for metadata namespaces?

As a note, for more fine-grained control over specific files, we have a top-level, per-file override key that allows you to override the specific value of a file option for any given file, for example:

override:
  PHP_NAMESPACE:
    acme/weather/v1/weather.proto: My\\Namespace\\Acme\\Weather\\V1
  PHP_METADATA_NAMESPACE:
    acme/weather/v1/weather.proto: Acme\\Weather\\V1\\Metadata

[prochac]

Hi, I have noticed you closed PR related to this issue https://github.com/bufbuild/buf/pull/1701

We are still at the beginning of gRPC adoption on PHP side, but we're again struggling with the namespace override. I will try to explain our use case.

We use the recommended package convention: The protofile is in /proto/companyXYZ/v1/file.proto.

syntax = "proto3";
package companyXYZ.v1;

With Go, we have go_package_prefix that identifies the dependency as gRPC pkg. In our case, we also use GitLab subdirs for better project structure. Go module name is: gitlab.com/companyXYZ/subdir-name/grpc-pkg-name/gen/proto/go And import path for the package is:

import companyXYZv1 "gitlab.com/companyXYZ/subdir-name/grpc-pkg-name/gen/proto/go/companyXYZ/v1"

The problem comes with PHP, where the namespace is CompanyXYZ\V1. That not signals that it's just gRPC domain dependency. Dirty tricks don't play well with PSR-4.

~Override in managed section doesn't seem to have any effect.~ Not like I would like to list all proto files there.

managed:
  override:
    PHP_NAMESPACE:
      companyXYZ/v1/file.proto: Company\gRPC\v1
    PHP_METADATA_NAMESPACE:
      companyXYZ/v1/file.proto: Company\gRPC\v1\Metadata

edit: it does work ... smth we can live so far with, but far from ideal. I overlooked git untracked files :sweat_smile:

[prochac]

Some kind of wildcard/globbing support would make my life a bit nicer tho

managed:
  override:
    PHP_NAMESPACE:
      companyXYZ/v1/*.proto: Company\gRPC\v1
    PHP_METADATA_NAMESPACE:
      companyXYZ/v1/*.proto: Company\gRPC\v1\Metadata

[prochac]

Does the new v2 buf.gen.yaml work this way?

after running buf config migrate all my overrides was changed to

version: v2
managed:
  enabled: true
  override:
    - file_option: php_metadata_namespace
      path: acme/weather/v1/weather.proto
      value: My\Namespace\Acme\Weather\V1
    - file_option: php_metadata_namespace
      path: acme/weather/v1/weather.proto
      value: My\Namespace\Acme\Weather\V1\Metadata

can the path be not finite? like just acme/weather/v1 ? @doriable

[doriable]

Does the new v2 buf.gen.yaml work this way?

can the path be not finite? like just acme/weather/v1 ?

Yep! So for v2, you can now specify a directory path for overrides and disables. Docs on overrides are available here: https://buf.build/docs/configuration/v2/buf-gen-yaml#override and we provide a sample buf.gen.yaml annotated with notes: https://buf.build/docs/configuration/v2/buf-gen-yaml

# When 'file_option', 'value', and 'path' are set, managed mode uses the value set
# in this rule instead of the default value for the specific file path. If the path
# is a directory, the rule affects all .proto files in the directory. Otherwise, it
# only affects the specified .proto file.

[doriable]

Just addressing this issue as a whole, we've added php_metadata_namespace_suffix as a part of the new managed mode options: https://buf.build/docs/generate/managed-mode#file-options

We've also reworked the way managed mode works in v2 of buf.gen.yaml, migrating to v2 will hopefully get folks to the desired disable/override behaviours for PHP: https://buf.build/docs/configuration/v2/buf-gen-yaml

[doriable]

Closing this for now since the above queries have been answered, please feel free to either re-open or create a new issue for any concerns., thanks!