[Integration][Harbor]: Add Harbor Ocean Integration

[ogubuikeAlex]

User description

Description

What - Created a new Ocean integration for Harbor that enables ingestion of Harbpr resources into Port's catalog. The integration supports:

• Project kind
• Artifact kind
• Repository kind
• User kind

Why - To enable users using Harbor to automatically sync their resources to Port

How:

• Core Features

  • Harbor API Client - Comprehensive async client for Harbor API v2.0
  • Support for projects, repositories, artifacts, users, and webhooks
  • Built-in pagination support for all list endpoints
  • Rate limiting and retry logic with exponential backoff
  • Configurable concurrent request management
  • SSL certificate verification support

• Real-time Webhook Support

  • Automatic webhook registration for all accessible projects
  • Support for artifact push, delete, and scan completion events
  • Support for project and repository lifecycle events
  • Webhook orchestration with permission checks

• Resource Synchronization

  • Full resync support for projects, users, repositories, and artifacts
  • Batch processing with configurable page sizes
  • Concurrent artifact fetching for improved performance
  • Automatic enrichment of artifacts with project and repository context

• Configuration Management

  • Flexible selectors for filtering resources
  • Project selector with query support and metadata inclusion
  • User selector with system user filtering
  • Repository selector with project-level filtering
  • Artifact selector with vulnerability and build history options
  • Configurable page sizes for all resource types

Type of change

Please leave one option from the following and delete the rest:

• [x] New Integration (non-breaking change which adds a new integration)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

• [x] Integration able to create all default resources from scratch
• [x] Resync finishes successfully
• [x] Resync able to create entities
• [x] Resync able to update entities
• [x] Resync able to detect and delete entities
• [ ] Scheduled resync able to abort existing resync and start a new one
• [ ] Tested with at least 2 integrations from scratch
• [x] Tested with Polling event listeners
• [ ] Tested deletion of entities that don't pass the selector

Integration testing checklist

• [x] Integration able to create all default resources from scratch
• [x] Completed a full resync from a freshly installed integration and it completed successfully
• [x] Resync able to create entities
• [x] Resync able to update entities
• [ ] Resync able to detect and delete entities
• [x] Resync finishes successfully
• [ ] If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
• [ ] If resource kind is updated, run the integration with the example data and check if the expected result is achieved
• [ ] If new resource kind is added or updated, validate that live-events for that resource are working as expected
• [x] Docs PR link #

Preflight checklist

• [x] Handled rate limiting
• [x] Handled pagination
• [x] Implemented the code in async
• [ ] Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.

---

PR Type

Enhancement, Tests

---

Description

• New Harbor Ocean Integration: Complete implementation of Harbor container registry integration for Port, enabling automatic synchronization of Harbor resources into Port's catalog

• Supported Resource Kinds: Projects, Users, Repositories, and Artifacts with comprehensive property mappings and relations

• Harbor API Client: Full-featured async client (1492 lines) with authentication, pagination, rate limiting, semaphore-based concurrency control, and webhook management

• Webhook Infrastructure: Complete webhook orchestration layer with processors for artifacts, repositories, and projects supporting real-time event handling (push, delete, pull, scanning, quota events)

• Comprehensive Testing: 653 lines of client unit tests and 560 lines of integration tests covering authentication, API operations, data transformation, webhook handling, and error scenarios

• Configuration & Blueprints: Port blueprints for all four resource types with JQ-based property mappings, selectors for resource filtering, and environment configuration

• Development Setup: Complete project structure with Poetry configuration, Makefile targets, pytest fixtures, and documentation

---

Diagram Walkthrough

flowchart LR
  Harbor["Harbor Registry"]
  Client["Harbor API Client"]
  Orchestrator["Webhook Orchestrator"]
  Processors["Event Processors<br/>Artifact/Repository/Project"]
  Port["Port Catalog"]
  
  Harbor -- "API Calls" --> Client
  Client -- "Webhook Setup" --> Orchestrator
  Harbor -- "Webhook Events" --> Processors
  Processors -- "Entity Sync" --> Port
  Client -- "Resync Data" --> Port

File Walkthrough

	Relevant files
Enhancement	10 files harbor_client.py Harbor API Client with Full Resource Management                    integrations/harbor/harbor/client/harbor_client.py Comprehensive Harbor API client implementation with 1492 lines supporting authentication, pagination, rate limiting, and webhook management Implements async methods for fetching projects, users, repositories, and artifacts with proper error handling and retry logic Includes semaphore-based concurrency control for managing concurrent requests and batch operations Provides webhook CRUD operations (create, update, delete) for project webhook policies +1492/-0 orchestrator.py Webhook Orchestration and Management Layer                              integrations/harbor/harbor/webhooks/orchestrator.py High-level webhook orchestration layer with 885 lines for managing Harbor webhooks across projects Implements setup, cleanup, and update operations for integration webhooks with permission checking Provides project-level webhook management with validation and error handling Includes comprehensive logging for all webhook operations and permission checks +885/-0  project_processor.py Project Webhook Event Processor                                                    integrations/harbor/harbor/webhooks/processors/project_processor.py Webhook processor for handling project-related events (quota events) Implements event validation, authentication, and payload processing Fetches latest project data when quota events are triggered Provides structured webhook event handling with proper error logging +87/-0    artifact_processor.py Artifact webhook event processor for Harbor integration    integrations/harbor/harbor/webhooks/processors/artifact_processor.py Implemented ArtifactWebhookProcessor class extending AbstractWebhookProcessor for handling artifact webhook events Processes Harbor events including PUSH_ARTIFACT, DELETE_ARTIFACT, PULL_ARTIFACT, and scanning-related events Extracts artifact information from webhook payloads and fetches updated artifact data from Harbor API Handles artifact deletion and distinguishes between deleted and updated artifacts in webhook results +156/-0  repository_processor.py Repository webhook event processor for Harbor integration integrations/harbor/harbor/webhooks/processors/repository_processor.py Implemented RepositoryWebhookProcessor class for handling repository-related webhook events Processes events affecting repositories including PUSH_ARTIFACT, DELETE_ARTIFACT, TAG_RETENTION, and REPLICATION Extracts repository metadata from webhook payloads and fetches updated repository data Handles repository deletion detection and empty repository scenarios +134/-0  main.py Integration entry point with resync and webhook handlers  integrations/harbor/main.py Defined resync handlers for all four resource kinds: projects, users, repositories, and artifacts Implemented on_start() lifecycle hook for Harbor connection validation and webhook setup Registered three webhook processors (ArtifactWebhookProcessor, ProjectWebhookProcessor, RepositoryWebhookProcessor) for real-time event handling Added comprehensive error handling and logging throughout resync and startup operations +128/-0  client_initializer.py Harbor client initialization with singleton pattern            integrations/harbor/harbor/client/client_initializer.py Implemented singleton pattern for Harbor client initialization with caching Retrieves Harbor configuration from get_harbor_config() function Returns cached client instance on subsequent calls to avoid redundant initialization Includes error handling for missing configuration +36/-0    integration.py Harbor integration base class and configuration                    integrations/harbor/integration.py Defined HarborIntegration class extending BaseIntegration as the main integration entry point Configured AppConfigHandlerClass to use APIPortAppConfig with HarborPortAppConfig as the configuration class Documented supported resource types: projects, users, repositories, and artifacts +26/-0    helper.py Helper function for building API request parameters            integrations/harbor/harbor/utils/helper.py Implemented build_params() function to construct API request parameters from event context Merges default page size with extra parameters and selector query filters Extracts query filters from resource configuration selectors when available +20/-0    validators.py Webhook signature verification utility                                      integrations/harbor/harbor/webhooks/validators.py Implemented verify_webhook_signature() function for HMAC-SHA256 signature verification Compares computed signature with provided signature using constant-time comparison Logs warning on signature verification failure +22/-0
Tests	3 files test_client.py Harbor Client Unit Tests                                                                  integrations/harbor/tests/test_client.py Comprehensive test suite with 653 lines covering Harbor client authentication and API operations Tests for projects, repositories, artifacts, and users endpoints with pagination and filtering Error handling tests including network timeouts, rate limiting, and invalid responses Request building and HTTP interaction tests with mocked responses +653/-0  test_harbor.py Harbor Integration Tests                                                                  integrations/harbor/tests/test_harbor.py Integration test suite with 560 lines covering end-to-end Harbor integration scenarios Tests for data transformation from Harbor resources to Port entities Webhook handling tests for push artifact and scan completion events Resync logic and error handling tests with various failure scenarios +560/-0  conftest.py Pytest configuration and test fixtures for Harbor integration integrations/harbor/tests/conftest.py Created comprehensive pytest configuration with session-scoped event loop fixture Implemented mock fixtures for Harbor client, HTTP client, and Port client with realistic response structures Added sample data fixtures for projects, repositories, artifacts, users, vulnerabilities, and webhook payloads Defined custom pytest markers for unit, integration, slow, webhook, and client test categorization +318/-0
Configuration changes	12 files app_config.py Harbor Integration Configuration and Resource Mappings      integrations/harbor/harbor/config/app_config.py Configuration module defining resource mappings for projects, users, repositories, and artifacts Implements HarborPortAppConfig with entity mappings and blueprint definitions for each resource kind Provides get_harbor_config() helper function to extract Harbor credentials from ocean config Defines selectors and port resource configurations for all Harbor entity types +187/-0  poetry.toml Poetry Virtual Environment Configuration                                  integrations/harbor/poetry.toml Poetry configuration for virtual environment management Enables in-project virtual environment creation +3/-0      selectors.py Configuration selectors for Harbor resource filtering        integrations/harbor/harbor/config/selectors.py Defined ProjectSelector with query filtering, metadata inclusion, and configurable page size Defined UserSelector with username filtering, system user inclusion, and page size options Defined RepositorySelector with project name filtering, query support, and pull count options Defined ArtifactSelector with comprehensive filtering options including vulnerability and build history inclusion +148/-0  constants.py Constants and enumerations for Harbor integration                integrations/harbor/harbor/constants.py Defined ObjectKind enum with four resource types: PROJECT, USER, REPOSITORY, ARTIFACT Listed supported webhook events including push, delete, pull, scanning, quota, replication, and retention events Defined constants for webhook naming, concurrent request limits, page size defaults, and client timeout +42/-0    blueprints.json Port blueprints for Harbor resource entities                          integrations/harbor/.port/resources/blueprints.json Defined four blueprints: harborProject, harborUser, harborRepository, harborArtifact Each blueprint includes comprehensive schema properties with appropriate data types and descriptions Configured relations between artifacts and repositories, and between repositories and projects Added vulnerability severity enums and artifact-specific properties for scan status and CVE counts +268/-0  port-app-config.yml Port application configuration with resource mappings        integrations/harbor/.port/resources/port-app-config.yml Defined resource mappings for four kinds: project, user, repository, artifact Implemented JQ-based property mappings from Harbor API responses to Port entity properties Configured entity identifiers, titles, and blueprint assignments for each resource type Added relation mappings between artifacts and repositories, and repositories and projects +130/-0  pyproject.toml Poetry project configuration and dependencies                        integrations/harbor/pyproject.toml Configured Poetry project metadata for Harbor integration version 0.1.0-beta Specified Python 3.12+ requirement with port_ocean as main dependency Added comprehensive dev dependencies including pytest, mypy, pylint, ruff, and black Configured towncrier for changelog management with breaking, deprecation, feature, improvement, bugfix, and doc categories +113/-0  MakeFile Makefile with development and deployment targets                  integrations/harbor/MakeFile Defined make targets for installation, linting, testing, and cleanup operations Implemented install, install/prod, install/local-core targets for dependency management Added lint, lint/fix targets for code quality checks using mypy, ruff, and black Included test, run, clean, and seed targets for development workflow +86/-0    py.ini Pytest configuration file for Harbor integration tests      integrations/harbor/tests/py.ini Configured pytest with test discovery patterns and Python 3.11+ minimum version Enabled verbose output, coverage reporting, parallel execution, and strict marker enforcement Defined custom markers for unit, integration, slow, webhook, client, and asyncio tests Configured mypy and coverage settings with appropriate exclusions and reporting options +112/-0  spec.yaml Harbor integration specification and configuration schema integrations/harbor/.port/spec.yaml Defined integration specification with Harbor icon and title Listed four exportable resource kinds: projects, users, repositories, artifacts Configured required parameters: harborUrl, harborUsername, harborPassword Added optional parameters for page size, webhook secret, app host, and various filter objects +104/-0  .env.example Example environment variables for Harbor integration          integrations/harbor/.env.example Provided example environment variables for Harbor integration configuration Included Port client credentials, integration identifier, and event listener settings Specified Harbor connection parameters: URL, username, password, page size, webhook secret Added optional filter configurations for projects, repositories, and artifacts +17/-0    sonar-project.properties SonarQube configuration for Harbor integration                      integrations/harbor/sonar-project.properties Configured SonarQube project key and organization for code quality analysis Set project identifier to port-labs_ocean_harbor-server under port-labs organization +2/-0
Miscellaneous	1 files debug.py Debug Entry Point                                                                                integrations/harbor/debug.py Simple debug entry point for running the Harbor integration locally Imports and executes the Ocean framework's run function +4/-0
Documentation	3 files README.md Harbor integration README with structure documentation      integrations/harbor/README.md Provided integration overview and documentation links Outlined directory structure with package organization for config, client, core, webhooks, and utils Documented file purposes and relationships within the integration architecture +41/-0    CHANGELOG.md Changelog for Harbor integration initial release                  integrations/harbor/CHANGELOG.md Documented initial release of Harbor integration version Listed supported resource types: users, projects, artifacts, repositories Followed Keep a Changelog format with semantic versioning +16/-0    CONTRIBUTING.md Contributing guidelines for Harbor integration                      integrations/harbor/CONTRIBUTING.md Provided contribution guidelines for Harbor integration development Included placeholder section for local setup instructions Noted importance of documenting gotchas like rate limiting and credential setup +7/-0
Formatting	1 files Makefile Minor formatting fix to root Makefile                                        Makefile Fixed trailing newline in file to ensure proper formatting +1/-1
Additional files	7 files __init__.py [link]    __init__.py [link]    __init__.py [link]    selectors.py [link]    __init__.py [link]    __init__.py [link]    __init__.py [link]

---