swift-unidoc icon indicating copy to clipboard operation
swift-unidoc copied to clipboard

support GitHub webhooks

Open tayloraswift opened this issue 1 year ago • 1 comments

crawling frequency is way too low for actively-developed packages with fewer than 50 stars, so we should support GitHub webhooks

  • [ ] whitelist GitHub IP addresses
  • [ ] determine if GitHub is capable of delivering webhooks over HTTP/2, and if not, add a POST carveout specifically for GitHub IP addresses
  • [ ] decode the create event and register it in the database
  • [ ] add a webhook secret to the package repo metadata

tayloraswift avatar May 18 '24 21:05 tayloraswift

it looks like GitHub delivers webhooks over HTTP/1.1, so a carveout is needed. however, it seems to be successfully surmounting the firewall already.

tayloraswift avatar May 18 '24 21:05 tayloraswift

i think for now, we can put off webhook secrets, as we are verifying almost all incoming webhook requests by IP address

tayloraswift avatar May 22 '24 21:05 tayloraswift