apache
Feature: Allow specifying 'scope' for Pulsar OAuth2 stream configuration
🐛 Problem Currently, when configuring a Pinot table to ingest from a Pulsar stream using OAuth2, it is not possible to specify the scope parameter.
The PulsarPartitionLevelConnectionHandler class initializes the connection using AuthenticationFactoryOAuth2.clientCredentials() but does not provide an argument for scope. This parameter was omitted in the original implementation (see PR #12068).
🎯 Motivation / Why this is needed
Compatibility with Identity Providers: Some OAuth2 issuers, like Ping Identity, require a scope to be present. If the scope field is null (or empty) and no default scope is configured on the identity server, the token retrieval request will fail. (See Ping Identity documentation on scopes).
Enforcing Required Scopes: Many secured environments mandate a specific, non-default scope (e.g., pulsar.read, api.access) for clients to receive a valid access token. Without the ability to specify this, Pinot cannot connect to Pulsar streams secured in this manner.
🛠️ Proposed Solution I propose adding a new, optional configuration property to support the OAuth2 scope.
Add new configuration key: Introduce an optional field, stream.pulsar.scope, to the streamConfigs map in the table configuration.
Propagate the configuration: Ensure this new scope value is read from the streamConfigs and made available to the Pulsar connection handler (e.g., by adding it to the PulsarConfig object).
Update connection handler: Modify the PulsarPartitionLevelConnectionHandler class. When building the Pulsar client, check if the scope value is provided. If it is, pass it to the AuthenticationFactoryOAuth2.clientCredentials(issuerUrl, credentialsUrl, audience, scope) method.
