discv4-dns-lists icon indicating copy to clipboard operation
discv4-dns-lists copied to clipboard

Feat/actions crawler

Open meowsbits opened this issue 5 years ago • 1 comments
trafficstars

Hey, We're running a downstream iteration of this at https://github.com/etclabscore/discv4-dns-lists, and have the crawler running on the Github Actions CI there. This frees up a devops task and makes the provenance of the lists transparent.

Here's a start at adding the same functionality here if you're interested.

The config currently uses Actions' schedule: cron config with what I thought is a sufficient time span. There are other ways to handle this, but this seemed the simplest to me.


TODO

Repo Secrets

The Action config depends on a few Secrets being set at the Github repo level.

  • [ ] CLOUDFLARE_API_TOKEN
  • [ ] ETH_DNS_CLOUDFLARE_ZONEID (This isn't necessary if you can get get the devp2p to-xxx to work without it.... I couldn't.)
  • [ ] ETH_DNS_DISCV4_KEY: The full content of the signing key file, eg. cat key.json | pbcopy or whatever.
  • [ ] ETH_DNS_DISCV4_KEYPASS: The key's password.
  • [ ] maybe a GITHUB_USER_PERSONAL_ACCESS_TOKEN (PAT). If you can get the git push to work without explicit user authentication (see comment below) then you won't need this.

meowsbits avatar Mar 04 '20 13:03 meowsbits

Hi, this looks nice, but we prefer running the crawler on our own infrastructure because that makes it easier to monitor it. Really appreciate the effort though.

fjl avatar Apr 29 '20 09:04 fjl