ocean icon indicating copy to clipboard operation
ocean copied to clipboard

Feat: BitBucket Intergration

Open Gahdloot opened this issue 10 months ago • 3 comments

User description

…ntication and webhook handling

Description

What - This pull request introduces a Bitbucket Cloud integration for Port's Ocean framework, allowing users to import and manage Bitbucket projects, repositories, pull requests, and components within Port.

Why -

  • Establishes a seamless connection between Bitbucket Cloud and Port.
  • Enables real-time updates through webhooks.
  • Optimizes performance with fully asynchronous API requests.
  • Incorporates rate-limit handling and pagination for improved stability.

How -

  • Utilizes Ocean’s asynchronous HTTP client for API interactions.
  • Implements authentication with the Bitbucket API using App Passwords.
  • Supports both scheduled and on-demand synchronization for precise data ingestion.
  • Handles webhook events to facilitate real-time updates.

Type of change

Please leave one option from the following and delete the rest:

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] New Integration (non-breaking change which adds a new integration)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] Non-breaking change (fix of existing functionality that will not change current behavior)
  • [ ] Documentation (added/updated documentation)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

  • [ ] Integration able to create all default resources from scratch
  • [ ] Resync finishes successfully
  • [ ] Resync able to create entities
  • [ ] Resync able to update entities
  • [ ] Resync able to detect and delete entities
  • [ ] Scheduled resync able to abort existing resync and start a new one
  • [ ] Tested with at least 2 integrations from scratch
  • [ ] Tested with Kafka and Polling event listeners
  • [ ] Tested deletion of entities that don't pass the selector

Integration testing checklist

  • [ ] Integration able to create all default resources from scratch
  • [ ] Resync able to create entities
  • [ ] Resync able to update entities
  • [ ] Resync able to detect and delete entities
  • [ ] Resync finishes successfully
  • [ ] If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
  • [ ] If resource kind is updated, run the integration with the example data and check if the expected result is achieved
  • [ ] If new resource kind is added or updated, validate that live-events for that resource are working as expected
  • [ ] Docs PR link here

Preflight checklist

  • [ ] Handled rate limiting
  • [ ] Handled pagination
  • [ ] Implemented the code in async
  • [ ] Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.


PR Type

Enhancement, Documentation, Tests


Description

  • Introduces Bitbucket Cloud integration for Port Ocean.

    • Implements authentication using Basic Auth with App Passwords.
    • Adds support for fetching projects, repositories, pull requests, and components.
    • Handles pagination and rate-limiting with exponential backoff.
  • Defines resource mappings and blueprints for Bitbucket entities.

  • Provides configuration and setup files for local development and testing.

  • Includes initial test cases and development tools setup.


Changes walkthrough 📝

Relevant files
Enhancement
6 files
client.py
Implements Bitbucket API client with authentication and data fetching
+132/-0 
debug.py
Adds debug entry point for integration                                     
+4/-0     
main.py
Implements resync handlers for Bitbucket resources             
+64/-0   
blueprints.json
Defines blueprints for Bitbucket resources                             
+77/-0   
port-app-config.yml
Maps Bitbucket resources to Port entities                               
+44/-0   
spec.yaml
Specifies integration metadata and configurations               
+28/-0   
Tests
1 files
test_sample.py
Adds a placeholder test case                                                         
+2/-0     
Configuration changes
6 files
launch.json
Adds debug configuration for Bitbucket integration             
+14/-1   
.env.example
Provides example environment variables for integration     
+6/-0     
Makefile
Links to shared Makefile for integration                                 
+1/-0     
poetry.toml
Configures virtual environment for integration                     
+3/-0     
pyproject.toml
Sets up project dependencies and tools                                     
+113/-0 
sonar-project.properties
Configures SonarQube project properties                                   
+2/-0     
Documentation
3 files
CHANGELOG.md
Adds changelog for Bitbucket integration                                 
+8/-0     
CONTRIBUTING.md
Adds contributing guidelines for integration                         
+7/-0     
README.md
Adds README for Bitbucket integration                                       
+7/-0     
Additional files
1 files
__init__.py [link]   

Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • Gahdloot avatar Feb 23 '25 14:02 Gahdloot

    Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 No relevant tests
    🔒 Security concerns

    Sensitive information exposure:
    The integration stores sensitive authentication credentials (username/password) in environment variables and generates Base64 encoded tokens. While Base64 encoding provides basic obfuscation, it's easily reversible. Consider using more secure authentication methods like OAuth tokens.

    ⚡ Recommended focus areas for review

    Initialization Order

    The BITBUCKET_API_BASE constant is used before it's defined, which could lead to AttributeError. The constant should be defined before being used in the init method.

        base_url=self.BITBUCKET_API_BASE,
        headers={
            "Authorization": f"Basic {self.auth_token}",
            "Content-Type": "application/json"
        }
    )
    self.BITBUCKET_API_BASE = "https://api.bitbucket.org/2.0"
    
    Variable Reference

    Multiple instances of self.WORKSPACE being referenced incorrectly as self.self.WORKSPACE which will cause AttributeError

    return await self._fetch_paginated_data(f"repositories/{self.self.WORKSPACE}/{repo_slug}/components")
    
    Missing Client Init

    BitbucketOceanIntegration is instantiated without required config parameter which will cause initialization error

    return BitbucketOceanIntegration().fetch_repositories()
    

    qodo-code-review[bot] avatar Feb 23 '25 14:02 qodo-code-review[bot]

    Qodo Merge was enabled for this repository. To continue using it, please link your Git account with your Qodo account here.

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Possible issue
    Fix variable initialization order
    Suggestion Impact:The BITBUCKET_API_BASE variable was moved before the client initialization as suggested, fixing the potential undefined variable issue

    code diff:

    +    def __init__(self):
    +        self.BITBUCKET_API_BASE = "https://api.bitbucket.org/2.0"
    +        self.WORKSPACE = ocean.integration_config.get("self.WORKSPACE")
             self.auth_token = BasicAuth.get_auth_token(
                 ocean.integration_config.get("username"), 
                 ocean.integration_config.get("password")
    @@ -45,9 +46,7 @@
                     "Authorization": f"Basic {self.auth_token}",
                     "Content-Type": "application/json"
                 }
    -        )
    -        self.BITBUCKET_API_BASE = "https://api.bitbucket.org/2.0"
    -        self.WORKSPACE = ocean.integration_config.get("self.WORKSPACE")
    +        )  
    

    The BITBUCKET_API_BASE URL is defined after it's used in the client
    initialization, which could cause undefined variable errors. Move the URL
    definition before the client initialization.

    integrations/bitbucket-cloud/bitbucket_integration/client.py [36-49]

     def __init__(self, config):
         super().__init__(config)
    +    self.BITBUCKET_API_BASE = "https://api.bitbucket.org/2.0"
         self.auth_token = BasicAuth.get_auth_token(
             ocean.integration_config.get("username"), 
             ocean.integration_config.get("password")
         )
         self.client = OceanAsyncClient(
             base_url=self.BITBUCKET_API_BASE,
             headers={
                 "Authorization": f"Basic {self.auth_token}",
                 "Content-Type": "application/json"
             }
         )
    -    self.BITBUCKET_API_BASE = "https://api.bitbucket.org/2.0"
    

    [Suggestion has been applied]

    Suggestion importance[1-10]: 9

    __

    Why: Critical bug fix - using BITBUCKET_API_BASE before it's defined would cause a runtime error, breaking the entire client initialization process.

    High
    Fix incorrect config key access
    Suggestion Impact:The commit refactored the code to use workspace instead of WORKSPACE and removed the self. prefix, addressing the incorrect config key access issue

    code diff:

    +    def __init__(self, workspace: str, token: str):
    +        self.workspace = workspace
    

    The self.WORKSPACE variable is incorrectly accessing the config with a self.
    prefix in the key, which will cause the workspace value to be None. Remove the
    self. prefix from the config key.

    integrations/bitbucket-cloud/bitbucket_integration/client.py [50]

    -self.WORKSPACE = ocean.integration_config.get("self.WORKSPACE")
    +self.WORKSPACE = ocean.integration_config.get("WORKSPACE")
    

    [Suggestion has been applied]

    Suggestion importance[1-10]: 8

    __

    Why: Important bug fix - the incorrect config key would cause the WORKSPACE value to be None, leading to failed API requests and non-functional integration.

    Medium
    Fix incorrect workspace path
    Suggestion Impact:The commit fixed the incorrect workspace path in fetch_components method by using self.workspace instead of self.self.WORKSPACE

    code diff:

    -        return await self._fetch_paginated_data(f"repositories/{self.self.WORKSPACE}/{repo_slug}/components")
    +        return await self._fetch_paginated_data(f"repositories/{self.workspace}/{repo_slug}/components")
    

    The fetch_components method uses an incorrect workspace path with double self.
    prefix, which will cause API requests to fail. Remove one of the self. prefixes.

    integrations/bitbucket-cloud/bitbucket_integration/client.py [77]

    -return await self._fetch_paginated_data(f"repositories/{self.self.WORKSPACE}/{repo_slug}/components")
    +return await self._fetch_paginated_data(f"repositories/{self.WORKSPACE}/{repo_slug}/components")
    

    [Suggestion has been applied]

    Suggestion importance[1-10]: 8

    __

    Why: Critical bug fix - the double self prefix in the API path would cause all component fetching operations to fail with invalid URL errors.

    Medium
    • [ ] Update

    qodo-code-review[bot] avatar Feb 23 '25 14:02 qodo-code-review[bot]

    This pull request is automatically being deployed by Amplify Hosting (learn more).

    Access this pull request here: https://pr-1422.d1ftd8v2gowp8w.amplifyapp.com