gapic-generator-python icon indicating copy to clipboard operation
gapic-generator-python copied to clipboard

feat: allow Callables for transport and channel init

Open daniel-sanche opened this issue 2 years ago • 7 comments

Our gapic libraries support passing in custom transport and channel objects when initializing clients and transports respectively.

In theory, this allows full customization of the transport stack to support uncommon or experimental use-cases, like channel pooling. In practice, customizing transports and channels in this way is messy, because the user needs to provide a set of parameters when constructing the transport or channel. These parameters depend on complex initialization logic (e.g., local credential parsing), which we already perform when a transport or channel is not provided. The user should not have to duplicate that code.

This change addresses the issue by allowing the user to provide a Callable in place of a custom transport or channel instance. The Callable will receive all the arguments that are already computed in the generator when the transport or channel is not provided, and is expected to return a usable instance (subclass of XXX or YYY).

We recommend that henceforth users generally provide such a callable rather than already instantiated transport or channel objects.


Example:

Instead of:

client_options = client_options_lib.ClientOptions()
api_endpoint, client_cert_source_func = self.get_mtls_endpoint_and_cert_source(client_options)
api_key_value = getattr(client_options, "api_key", None)

custom_transport = MyCustomTransport(
  credentials=None,
  credentials_file=client_options.credentials_file,
  host=api_endpoint,
  scopes=client_options.scopes,
  client_cert_source_for_mtls=client_cert_source_func,
  quota_project_id=client_options.quota_project_id,
  client_info=DEFAULT_CLIENT_INFO,
  always_use_jwt_access=True,
  api_audience=client_options.api_audience,
)
client = BigtableAsyncClient(transport=custom_transport)

We can do:

client = BigtableAsyncClient(transport=MyCustomTransport)

or:

client = BigtableAsyncClient(transport=lambda *args, **kwargs: MyCustomTransport(*args,  pool_size=5, **kwargs))

Let me know if you have any feedback. If we decide to go forward with his, I can add some tests and polish

daniel-sanche avatar Jul 17 '23 21:07 daniel-sanche

🤖 I detect that the PR title and the commit message differ and there's only one commit. To use the PR title for the commit history, you can use Github's automerge feature with squashing, or use automerge label. Good luck human!

-- conventional-commit-lint bot https://conventionalcommits.org/

Converting to draft as tests are failing. Please convert back to review once tests are passing.

parthea avatar Sep 02 '23 10:09 parthea

tests are now passing

daniel-sanche avatar Sep 08 '23 21:09 daniel-sanche

Marking as draft as presubmits are failing

parthea avatar Nov 27 '23 18:11 parthea

fixed presubmits

daniel-sanche avatar Feb 16 '24 00:02 daniel-sanche

LGTM, but please wait for review from @vchudnov-g

parthea avatar Feb 21 '24 11:02 parthea

@vchudnov-g Please can you take a look?

parthea avatar Mar 20 '24 18:03 parthea