ocean icon indicating copy to clipboard operation
ocean copied to clipboard

[Integration][Github] added github integration to ocean

Open Lanrey opened this issue 7 months ago • 3 comments
trafficstars

User description

Description

What - Added github integration to ocean

Why - To sync some github resources via the ocean framework

How - Used webhooks to listen for github events, and ocean's httpx async custom package to fetch the data from github api

Type of change

Please leave one option from the following and delete the rest:

  • [x ] New Integration (non-breaking change which adds a new integration)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

  • [ x] Integration able to create all default resources from scratch
  • [x ] Resync finishes successfully
  • [x ] Resync able to create entities
  • [ x] Resync able to update entities
  • [x ] Resync able to detect and delete entities
  • [ x] Scheduled resync able to abort existing resync and start a new one
  • [x ] Tested with at least 2 integrations from scratch
  • [ x] Tested with Kafka and Polling event listeners
  • [ x] Tested deletion of entities that don't pass the selector

Integration testing checklist

  • [x ] Integration able to create all default resources from scratch
  • [x ] Resync able to create entities
  • [x ] Resync able to update entities
  • [ x] Resync able to detect and delete entities
  • [ x] Resync finishes successfully
  • [x ] If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
  • [ x] If resource kind is updated, run the integration with the example data and check if the expected result is achieved
  • [x ] If new resource kind is added or updated, validate that live-events for that resource are working as expected
  • [x ] Docs PR link here

Preflight checklist

  • [x ] Handled rate limiting
  • [ x] Handled pagination
  • [x ] Implemented the code in async
  • [ x] Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

Screenshot-2025-04-16-at-13-55-54-2 Screenshot-2025-04-16-at-13-56-00-2 Screenshot-2025-04-16-at-13-56-05-2

API Documentation

Provide links to the API documentation used for this integration.


PR Type

New Integration, Enhancement, Tests


Description

  • Added a new GitHub integration to Ocean for syncing GitHub resources.

  • Implemented webhook processors for GitHub events (issues, pull requests, teams, workflows, etc.).

  • Developed extensive test cases for the GitHub client and webhook processors.

  • Created configuration files for integration setup, including .env.example, blueprints.json, and port-app-config.yaml.


Changes walkthrough 📝

Relevant files
Enhancement
13 files
debug.py
Add debug entry point for GitHub integration                         
+4/-0     
client.py
Implement GitHub client for API communication                       
+298/-0 
overrides.py
Define resource configurations and selectors for GitHub   
+87/-0   
initialize_client.py
Add helper to initialize GitHub client                                     
+16/-0   
integration.py
Define base integration class for GitHub                                 
+9/-0     
kinds.py
Define enumeration for GitHub entity kinds                             
+15/-0   
main.py
Main entry point for GitHub integration                                   
+114/-0 
issue_webhook_processor.py
Add processor for GitHub issue webhook events                       
+86/-0   
pull_request_webhook_processor.py
Add processor for GitHub pull request webhook events         
+83/-0   
push_webhook_processor.py
Add processor for GitHub push webhook events                         
+150/-0 
repository_webhook_processor.py
Add processor for GitHub repository webhook events             
+95/-0   
team_webhook_processor.py
Add processor for GitHub team webhook events                         
+80/-0   
workflow_webhook_processor.py
Add processor for GitHub workflow run webhook events         
+78/-0   
Tests
3 files
test_client.py
Add test cases for GitHub client                                                 
+479/-0 
test_processor.py
Add test cases for webhook processors                                       
+249/-0 
test_sample.py
Add placeholder test example                                                         
+2/-0     
Configuration changes
6 files
.env.example
Provide example environment variables for GitHub integration
+6/-0     
blueprints.json
Define blueprints for GitHub resources in Port                     
+429/-0 
port-app-config.yaml
Add Port app configuration for GitHub integration               
+179/-0 
spec.yaml
Add specification file for GitHub integration                       
+58/-0   
Makefile
Add Makefile for GitHub integration setup                               
+1/-0     
sonar-project.properties
Add SonarQube configuration for GitHub integration             
+2/-0     
Documentation
3 files
CHANGELOG.md
Add changelog for GitHub integration                                         
+42/-0   
CONTRIBUTING.md
Add contributing guidelines for GitHub integration             
+7/-0     
README.md
Add README for GitHub integration                                               
+7/-0     
Dependencies
2 files
poetry.toml
Add Poetry configuration for virtual environments               
+3/-0     
pyproject.toml
Define project dependencies and configurations                     
+113/-0 
Additional files
3 files
__init__.py [link]   
__init__.py [link]   
__init__.py [link]   

Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • Lanrey avatar Apr 11 '25 06:04 Lanrey