[Integration][Bitbucket server] Improve resync performance for large deployments

[dev-habib-nuhu]

User description

Description

What

• Added configurable performance optimizations (page size, concurrent requests, rate limiting, project filtering) to the Bitbucket Server integration.

Why

• Bitbucket Server Integration takes a long time before completing resync

How

• Implemented semaphore-controlled parallel PR fetching, configurable pagination, flexible rate limits, and regex/suffix project filtering.

Type of change

• [x] New feature (non-breaking change which adds functionality)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

• [ ] Integration able to create all default resources from scratch
• [ ] Resync finishes successfully
• [ ] Resync able to create entities
• [ ] Resync able to update entities
• [ ] Resync able to detect and delete entities
• [ ] Scheduled resync able to abort existing resync and start a new one
• [ ] Tested with at least 2 integrations from scratch
• [ ] Tested with Kafka and Polling event listeners
• [ ] Tested deletion of entities that don't pass the selector

Integration testing checklist

• [ ] Integration able to create all default resources from scratch
• [ ] Completed a full resync from a freshly installed integration and it completed successfully
• [ ] Resync able to create entities
• [ ] Resync able to update entities
• [ ] Resync able to detect and delete entities
• [ ] Resync finishes successfully
• [ ] If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
• [ ] If resource kind is updated, run the integration with the example data and check if the expected result is achieved
• [ ] If new resource kind is added or updated, validate that live-events for that resource are working as expected
• [ ] Docs PR link here

Preflight checklist

• [ ] Handled rate limiting
• [ ] Handled pagination
• [ ] Implemented the code in async
• [ ] Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.

---

PR Type

Enhancement

---

Description

• Add 6 configurable performance optimization parameters to Bitbucket Server integration

• Implement parallel PR fetching with semaphore-controlled concurrency

• Add project filtering by regex patterns and suffix matching

• Make pagination page size and rate limiting configurable

---

Diagram Walkthrough

flowchart LR
  Config["Configuration Parameters"]
  Config -->|rate_limit, rate_limit_window| RateLimiter["AsyncLimiter"]
  Config -->|page_size| Pagination["Paginated Requests"]
  Config -->|max_concurrent_requests| Semaphore["BoundedSemaphore"]
  Config -->|projects_filter_regex, suffix| ProjectFilter["Project Filtering"]
  Semaphore -->|controls| PRFetching["Parallel PR Fetching"]
  ProjectFilter -->|filters| Projects["Project Batches"]
  Pagination -->|fetches| Resources["API Resources"]
  RateLimiter -->|throttles| Resources

File Walkthrough

	Relevant files
Enhancement	client.py Add configurable performance optimizations and project filtering integrations/bitbucket-server/client.py Added 6 new optional configuration parameters (rate_limit, rate_limit_window, page_size, max_concurrent_requests, projects_filter_regex, projects_filter_suffix) Implemented _should_include_project() method for regex and suffix-based project filtering Added asyncio.BoundedSemaphore for controlling concurrent PR requests Modified get_paginated_resource() to use configurable page size Updated get_projects() to apply regex/suffix filters to project batches Modified get_pull_requests() to use semaphore-controlled parallel fetching with semaphore_async_iterator +90/-14  utils.py Update client initialization with new config parameters    integrations/bitbucket-server/utils.py Updated initialize_client() to extract and pass all 6 new configuration parameters Added default value handling for rate limiting, pagination, and concurrency settings Refactored config access to use local variable for cleaner code +36/-6    webhook_client.py Update webhook client initialization with config parameters integrations/bitbucket-server/webhook_processors/webhook_client.py Updated initialize_client() function to extract and pass all 6 new configuration parameters Added default value handling matching the main client initialization Refactored config access pattern for consistency +25/-5
Tests	test_client.py Add comprehensive tests for performance optimizations        integrations/bitbucket-server/tests/test_client.py Added 13 new test cases covering configurable page size, rate limits, and concurrency Added tests for project filtering with regex patterns, suffix patterns, and combined filters Added tests verifying default values and disabled filtering behavior Added test for semaphore-based concurrency control in PR fetching +212/-2
Configuration changes	spec.yaml Add configuration spec for performance optimization parameters integrations/bitbucket-server/.port/spec.yaml Added 6 new optional configuration fields to spec Defined bitbucketRateLimit (default: 1000) and bitbucketRateLimitWindow (default: 3600) Defined bitbucketPageSize (default: 25) and bitbucketMaxConcurrentRequests (default: 10) Added bitbucketProjectsFilterRegex and bitbucketProjectsFilterSuffix for project filtering +28/-0    pyproject.toml Bump version to 0.1.63-beta                                                            integrations/bitbucket-server/pyproject.toml Bumped version from 0.1.62-beta to 0.1.63-beta +1/-1
Documentation	CHANGELOG.md Update changelog with version 0.1.63-beta release                integrations/bitbucket-server/CHANGELOG.md Added version 0.1.63-beta release notes Documented bug fix for long resync completion times +7/-0

---