ocean icon indicating copy to clipboard operation
ocean copied to clipboard

Feature/bitbucket ocean Add initial Bitbucket integration with authe…

Open Gahdloot opened this issue 10 months ago • 3 comments

User description

…ntication and webhook handling

Description

What - This pull request introduces a Bitbucket Cloud integration for Port's Ocean framework, allowing users to import and manage Bitbucket projects, repositories, pull requests, and components within Port.

Why -

  • Establishes a seamless connection between Bitbucket Cloud and Port.
  • Enables real-time updates through webhooks.
  • Optimizes performance with fully asynchronous API requests.
  • Incorporates rate-limit handling and pagination for improved stability.

How -

  • Utilizes Ocean’s asynchronous HTTP client for API interactions.
  • Implements authentication with the Bitbucket API using App Passwords.
  • Supports both scheduled and on-demand synchronization for precise data ingestion.
  • Handles webhook events to facilitate real-time updates.

Type of change

Please leave one option from the following and delete the rest:

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] New Integration (non-breaking change which adds a new integration)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] Non-breaking change (fix of existing functionality that will not change current behavior)
  • [ ] Documentation (added/updated documentation)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

  • [ ] Integration able to create all default resources from scratch
  • [ ] Resync finishes successfully
  • [ ] Resync able to create entities
  • [ ] Resync able to update entities
  • [ ] Resync able to detect and delete entities
  • [ ] Scheduled resync able to abort existing resync and start a new one
  • [ ] Tested with at least 2 integrations from scratch
  • [ ] Tested with Kafka and Polling event listeners
  • [ ] Tested deletion of entities that don't pass the selector

Integration testing checklist

  • [ ] Integration able to create all default resources from scratch
  • [ ] Resync able to create entities
  • [ ] Resync able to update entities
  • [ ] Resync able to detect and delete entities
  • [ ] Resync finishes successfully
  • [ ] If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
  • [ ] If resource kind is updated, run the integration with the example data and check if the expected result is achieved
  • [ ] If new resource kind is added or updated, validate that live-events for that resource are working as expected
  • [ ] Docs PR link here

Preflight checklist

  • [ ] Handled rate limiting
  • [ ] Handled pagination
  • [ ] Implemented the code in async
  • [ ] Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.


PR Type

Enhancement, Tests


Description

  • Introduced a new Bitbucket integration for Port's Ocean framework.

  • Implemented authentication using Basic Auth with App Passwords.

  • Added support for fetching projects, repositories, and pull requests.

  • Included webhook handling and data ingestion into Port.

  • Provided comprehensive unit tests for authentication, integration, and webhook handling.


Changes walkthrough 📝

Relevant files
Enhancement
4 files
auth.py
Added Basic Authentication logic for Bitbucket API             
+24/-0   
integration.py
Implemented core Bitbucket integration with data fetching
+104/-0 
webhook.py
Added webhook handling for Bitbucket events                           
+6/-0     
main.py
Created entry point for Bitbucket integration execution   
+17/-0   
Configuration changes
3 files
config.py
Added configuration management for Bitbucket integration 
+44/-0   
Makefile
Linked infrastructure Makefile for Bitbucket integration 
+1/-0     
pytest.ini
Configured pytest for Bitbucket integration tests               
+6/-0     
Tests
3 files
test_auth.py
Added unit tests for authentication logic                               
+29/-0   
test_integration.py
Added unit tests for Bitbucket integration methods             
+43/-0   
test_webhook.py
Added unit tests for webhook event handling                           
+52/-0   
Dependencies
1 files
pyproject.toml
Added project metadata and dependencies for Bitbucket integration
+20/-0   
Additional files
1 files
__init__.py [link]   

Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • Gahdloot avatar Feb 20 '25 13:02 Gahdloot

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 PR contains tests
    🔒 Security concerns

    Sensitive information exposure:
    The integration logs debug information that could potentially expose sensitive data, particularly in the _fetch_paginated_data method where the entire API response is logged (line 66 in integration.py). This could inadvertently expose sensitive repository or user information in logs.

    ⚡ Recommended focus areas for review

    Error Handling

    The _fetch_paginated_data method catches all exceptions generically and continues after max retries. This could mask critical errors and lead to incomplete data ingestion.

    except Exception as e:
        if retries < max_retries:
            wait_time = 2 ** retries  # Exponential backoff (2s, 4s, 8s)
            logger.warning(f"Request error, retrying in {wait_time} seconds... Error: {e}")
            await asyncio.sleep(wait_time)
            retries += 1
        else:
            logger.error(f"Max retries exceeded for {endpoint}. Skipping...")
            break
    
    Config Validation

    The config only logs warnings for missing required environment variables but continues execution. This could lead to runtime errors when these values are actually needed.

    if not workspace:
        logger.warning("BITBUCKET_WORKSPACE is not set.")
    if not username:
        logger.warning("BITBUCKET_USERNAME is not set.")
    if not app_password:
        logger.warning("BITBUCKET_APP_PASSWORD is not set.")
    
    Invalid Package Name

    The package name contains invalid characters (**) which will cause issues with package installation and distribution.

    name = "bit_bucket-**"
    

    qodo-code-review[bot] avatar Feb 20 '25 13:02 qodo-code-review[bot]

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Security
    Add input sanitization for security

    Add input sanitization to prevent potential injection attacks in the
    authentication token generation by stripping or escaping special characters.

    integrations/bit_bucket/bit_bucket_integration/auth.py [22]

    -auth_token = base64.b64encode(f"{username}:{password}".encode()).decode()
    +sanitized_username = str(username).strip()
    +sanitized_password = str(password).strip()
    +auth_token = base64.b64encode(f"{sanitized_username}:{sanitized_password}".encode()).decode()
    
    • [ ] Apply this suggestion
    Suggestion importance[1-10]: 8

    __

    Why: The suggestion addresses a potential security vulnerability by sanitizing authentication inputs, which is crucial for preventing injection attacks in authentication tokens.

    Medium
    General
    Add request timeout protection

    Add a timeout parameter to the client.get() request to prevent indefinite
    hanging in case of network issues.

    integrations/bit_bucket/bit_bucket_integration/integration.py [62]

    -response = await self.client.get(url)
    +response = await self.client.get(url, timeout=30)  # 30 seconds timeout
    
    • [ ] Apply this suggestion
    Suggestion importance[1-10]: 8

    __

    Why: Adding a timeout parameter is critical for preventing the application from hanging indefinitely during network issues, which could impact system stability and resource usage.

    Medium
    Possible issue
    Improve API error handling

    Add response status code validation before processing the response data to
    prevent processing invalid responses. Check if status code is in the 200-299
    range.

    integrations/bit_bucket/bit_bucket_integration/integration.py [62-66]

     response = await self.client.get(url)
    -response.raise_for_status()
    +if not 200 <= response.status < 300:
    +    logger.error(f"API request failed with status {response.status}: {await response.text()}")
    +    response.raise_for_status()
     
     data = await response.json()
     logger.debug(f"API Response: {data}")
    
    • [ ] Apply this suggestion
    Suggestion importance[1-10]: 7

    __

    Why: The suggestion adds valuable error logging with response details before raising the exception, which would help in debugging API issues. The status code check provides more detailed error information.

    Medium
    • [ ] More

    qodo-code-review[bot] avatar Feb 20 '25 13:02 qodo-code-review[bot]

    This pull request is automatically being deployed by Amplify Hosting (learn more).

    Access this pull request here: https://pr-1415.d1ftd8v2gowp8w.amplifyapp.com