[Integration][Bitbucket] Added multiple token support

[oiadebayo]

User description

Description

What - Added support for multiple token usage in bitbucket client for heavily limited API

Why - To increase efficiency of the integration and reduce wait time after processing

How - I added a manager class that handles the rotation between multiple client instance using different tokens for auth. The clients are mannaged with priority queues and prioritised based on availability.

Type of change

Please leave one option from the following and delete the rest:

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] New Integration (non-breaking change which adds a new integration)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] Non-breaking change (fix of existing functionality that will not change current behavior)
• [ ] Documentation (added/updated documentation)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

• [ ] Integration able to create all default resources from scratch
• [ ] Resync finishes successfully
• [ ] Resync able to create entities
• [ ] Resync able to update entities
• [ ] Resync able to detect and delete entities
• [ ] Scheduled resync able to abort existing resync and start a new one
• [ ] Tested with at least 2 integrations from scratch
• [ ] Tested with Kafka and Polling event listeners
• [ ] Tested deletion of entities that don't pass the selector

Integration testing checklist

• [ ] Integration able to create all default resources from scratch
• [ ] Resync able to create entities
• [ ] Resync able to update entities
• [ ] Resync able to detect and delete entities
• [ ] Resync finishes successfully
• [ ] If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
• [ ] If resource kind is updated, run the integration with the example data and check if the expected result is achieved
• [ ] If new resource kind is added or updated, validate that live-events for that resource are working as expected
• [ ] Docs PR link here

Preflight checklist

• [ ] Handled rate limiting
• [ ] Handled pagination
• [ ] Implemented the code in async
• [ ] Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.

---

PR Type

Enhancement, Tests

---

Description

• Added support for multiple token usage in Bitbucket integration.

• Introduced BitbucketClientManager for token rotation and rate limiting.

• Replaced rate-limited paginated API calls with token-based management.

• Updated tests to validate multiple token functionality.

---

Changes walkthrough 📝

	Relevant files
Enhancement	5 files client.py Refactored Bitbucket client for multiple token support.    +31/-46  cache.py Added coroutine result caching utility.                                    +41/-0    multiple_token.py Introduced `BitbucketClientManager` for token rotation.    +127/-0  rate_limiter.py Enhanced rate limiter with capacity and availability checks. +31/-0    main.py Integrated `BitbucketClientManager` into resync workflows. +25/-10
Tests	1 files test_client.py Updated tests for multiple token and client manager.          +23/-26
Configuration changes	1 files spec.yaml Updated configuration to support multiple credentials.      +3/-9

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-code-review[bot]]

Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 Security concerns Sensitive information exposure: The PR adds code that handles multiple authentication tokens, but there's a potential issue in the client.py file where credentials are split and parsed (lines 54-60). The code only takes the first credential from a comma-separated list, which could lead to unexpected authentication behavior. Additionally, error messages in multiple_token.py (line 124) might leak sensitive information about which client ID failed, potentially revealing information about your token rotation strategy.

⚡ Recommended focus areas for review Error Handling The execute_request method catches all exceptions but only handles 429 status codes. Other HTTP errors or exceptions might need specific handling or logging strategies. while True: client_id, client, limiter = await self._rotate_client() try: async with limiter: client_method = getattr(client, method_name) async for item in client_method(*args, **kwargs): yield item break except Exception as e: if hasattr(e, "status_code") and e.status_code == 429: logger.warning( f"Rate limit hit for client {client_id}, rotating to next client" ) continue logger.error(f"Error while making request with {client_id}: {e}") Credential Parsing The create_from_ocean_config method splits credentials on commas and only uses the first one, which is inconsistent with the multiple token support implementation. credentials = ocean.integration_config.get("bitbucket_credentials") if not credentials: raise MissingIntegrationCredentialException( "Either workspace token or both username and app password must be provided" ) credentials = credentials.split(",")[0] username, app_password = ( Cache Implementation The cache_coroutine_result decorator is used on init_multiple_client but not on init_client, which could lead to inconsistent caching behavior. @cache_coroutine_result() async def init_multiple_client() -> BitbucketClientManager: return BitbucketClientManager( workspace=ocean.integration_config["bitbucket_workspace"], host=ocean.integration_config["bitbucket_host_url"], credentials=ocean.integration_config["bitbucket_credentials"], limit_per_client=BitbucketRateLimiterConfig.LIMIT, window=BitbucketRateLimiterConfig.WINDOW, )

[qodo-code-review[bot]]

Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	✅ Fix client rotation logic Suggestion Impact: The suggestion was implemented in the commit. The break statement was moved from being inside the async with block to being after it, which ensures proper client rotation after successfully yielding all items from a client method. code diff: - yield item - break The current implementation will break the loop after successfully yielding all items from a client method, which means it won't rotate to another client if the first one succeeds. This could lead to uneven load distribution across clients. Consider moving the break statement inside the try block after the generator loop completes to ensure proper client rotation. integrations/bitbucket-cloud/helpers/multiple_token.py [102-127] async def execute_request( self, method_name: str, *args: Any, **kwargs: Any ) -> AsyncGenerator[Any, None]: """ Execute a method on a rotated BitbucketClient instance with precise rate limiting. If the client method is an async generator, yield items as they are produced. Will rotate to another client only on rate limit (429) errors, raises all other exceptions. """ while True: client_id, client, limiter = await self._rotate_client() try: async with limiter: client_method = getattr(client, method_name) async for item in client_method(*args, **kwargs): yield item - break + break # Only break after successfully completing the generator except Exception as e: if hasattr(e, "status_code") and e.status_code == 429: logger.warning( f"Rate limit hit for client {client_id}, rotating to next client" ) continue logger.error(f"Error while making request with {client_id}: {e}") raise [Suggestion has been applied] Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies a logical error in the client rotation implementation. Moving the break statement outside the async with block ensures all items are yielded before breaking the loop, which prevents premature termination and improves load distribution across clients.	Medium
	Handle missing event context The current implementation doesn't handle the case where event.attributes might be None, which could happen if the event context isn't properly initialized. This could lead to AttributeError exceptions. Add a check to ensure the event context is available before accessing attributes. integrations/bitbucket-cloud/helpers/cache.py [9-41] def cache_coroutine_result() -> Callable[[AsyncCallable], AsyncCallable]: """Coroutine version of `cache_iterator_result` from port_ocean.utils.cache Decorator that caches the result of a coroutine function. It checks if the result is already in the cache, and if not, fetches the result, caches it, and returns the cached value. The cache is stored in the scope of the running event and is removed when the event is finished. Usage: ```python @cache_coroutine_result() async def my_coroutine_function(): # Your code here ``` """ def decorator(func: AsyncCallable) -> AsyncCallable: @functools.wraps(func) async def wrapper(*args: Any, **kwargs: Any) -> Any: + # Check if event context is available + if not hasattr(event, 'attributes') or event.attributes is None: + return await func(*args, **kwargs) + cache_key = hash_func(func.__name__, *args, **kwargs) if cache := event.attributes.get(cache_key): return cache result = await func(*args, **kwargs) event.attributes[cache_key] = result return result return wrapper return decorator [ ] Apply this suggestion Suggestion importance[1-10]: 8 __ Why: The suggestion addresses a potential runtime error by checking if the event context is properly initialized before accessing its attributes. This defensive programming approach prevents AttributeError exceptions when the cache decorator is used outside an event context.	Medium
General	Simplify credential parsing logic The condition if credentials and not username and not app_password: is redundant since we already check if "::" in credentials to determine if it's a username/password pair. If it's not, we assume it's a workspace token. This could lead to confusing behavior if the credential string is malformed. Simplify the logic to make it more robust. integrations/bitbucket-cloud/client.py [51-80] @classmethod def create_from_ocean_config(cls) -> "BitbucketClient": """Create a BitbucketClient from the Ocean config.""" credentials = ocean.integration_config.get("bitbucket_credentials") if not credentials: raise MissingIntegrationCredentialException( "Either workspace token or both username and app password must be provided" ) credentials = credentials.split(",")[0] - username, app_password = ( - credentials.split("::") if "::" in credentials else (None, None) - ) - if credentials and not username and not app_password: - workspace_token = credentials - return cls( - workspace=ocean.integration_config["bitbucket_workspace"], - host=ocean.integration_config["bitbucket_host_url"], - workspace_token=workspace_token, - ) - elif username and app_password: + + if "::" in credentials: + username, app_password = credentials.split("::", 1) return cls( workspace=ocean.integration_config["bitbucket_workspace"], host=ocean.integration_config["bitbucket_host_url"], username=username, app_password=app_password, ) else: - raise MissingIntegrationCredentialException( - "Either workspace token or both username and app password must be provided" + # If no "::" separator, treat as workspace token + workspace_token = credentials + return cls( + workspace=ocean.integration_config["bitbucket_workspace"], + host=ocean.integration_config["bitbucket_host_url"], + workspace_token=workspace_token, ) [ ] Apply this suggestion Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies redundant logic in credential parsing that could lead to confusion. The improved code simplifies the flow by directly checking for the separator and handling each case more clearly, making the code more maintainable and less error-prone.	Medium
[ ] Update

[aws-amplify-eu-west-1[bot]]

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-1511.d1ftd8v2gowp8w.amplifyapp.com

[qodo-code-review[bot]]

Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

CI Feedback 🧐

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: 🚢 bitbucket-cloud

Failed stage: Lint [❌]

Failed test name: make lint

Failure summary: The action failed during the make lint step with the following issues: 1. Mypy errors in bitbucket_cloud/helpers/multiple_token.py: - Cannot find implementation or library stub for module helpers.rate_limiter (line 5) - Cannot find implementation or library stub for module client (line 6) 2. Ruff (linter) errors: - Unused import BitbucketRateLimiterConfig in bitbucket_cloud/helpers/utils - Unused import BitbucketClient in main.py (line 8) - One more unused import error (3 total fixable errors)

Relevant error logs: 1: ##[group]Operating System 2: Ubuntu ... 1066: Installing the current project: bitbucket-cloud (0.1.3-dev) 1067: ##[group]Run make lint 1068: [36;1mmake lint[0m 1069: shell: /usr/bin/bash -e {0} 1070: env: 1071: pythonLocation: /opt/hostedtoolcache/Python/3.12.9/x64 1072: PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.9/x64/lib/pkgconfig 1073: Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.9/x64 1074: Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.9/x64 1075: Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.9/x64 1076: LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.9/x64/lib 1077: ##[endgroup] 1078: Running poetry check 1079: All set! 1080: Running mypy 1081: bitbucket_cloud/helpers/multiple_token.py:5: error: Cannot find implementation or library stub for module named "helpers.rate_limiter" [import-not-found] 1082: bitbucket_cloud/helpers/multiple_token.py:5: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports 1083: bitbucket_cloud/helpers/multiple_token.py:6: error: Cannot find implementation or library stub for module named "client" [import-not-found] 1084: Found 2 errors in 1 file (checked 32 source files) 1085: Running ruff ... 1102: 9 | from bitbucket_cloud.helpers.utils import BitbucketRateLimiterConfig 1103: | ^^^^^^^^^^^^^^^^^^^^^^^^^^ F401 1104: 10 | import base64 1105: | 1106: = help: Remove unused import: `bitbucket_cloud.helpers.utils.BitbucketRateLimiterConfig` 1107: main.py:8:36: F401 [*] `bitbucket_cloud.client.BitbucketClient` imported but unused 1108: | 1109: 6 | from port_ocean.core.ocean_types import ASYNC_GENERATOR_RESYNC_TYPE 1110: 7 | from port_ocean.utils.async_iterators import stream_async_iterators_tasks 1111: 8 | from bitbucket_cloud.client import BitbucketClient 1112: | ^^^^^^^^^^^^^^^ F401 1113: 9 | from bitbucket_cloud.helpers.multiple_token import BitbucketClientManager 1114: 10 | from bitbucket_cloud.helpers.utils import ObjectKind 1115: | 1116: = help: Remove unused import: `bitbucket_cloud.client.BitbucketClient` 1117: Found 3 errors. 1118: [*] 3 fixable with the `--fix` option. 1119: Running black 1120: All done! ✨ 🍰 ✨ 1121: 32 files would be left unchanged. 1122: Running yamllint 1123: [0;31mOne or more checks failed with exit code 1[0m 1124: make: *** [../_infra/Makefile:62: lint] Error 1 1125: ##[error]Process completed with exit code 2. 1126: Post job cleanup.