register icon indicating copy to clipboard operation
register copied to clipboard

Moving to another DNS provider

Open wdhdev opened this issue 1 year ago • 40 comments

(reference #13995)

Alternatives

Cloudflare

  • Pros
    • Free, up until certain limits
    • Cloudflare proxy option
    • Caching via Cloudflare's global CDN (when proxy is enabled)
    • Faster DNS deployment
    • Built-in DDoS protection for subdomains (when proxy is enabled)
    • NS record support for subdomains
    • Free email forwarding (can setup [email protected], [email protected], etc)
    • Cloudflare Zero Trust
      • Tunnels
  • Cons
    • Record limit, although can be increased by contacting support I believe
    • Not great support

Bunny.net

  • Pros
    • Good support
    • Custom/whitelabel nameserver support (e.g. ns1.is-a.dev & ns2.is-a.dev show as the nameserver records, but Bunny.net is the backend)
    • DNS query logs and statistics
    • Importing zone file
  • Cons
    • Paid
    • Limit of 20 million queries a month, before it's paid image
    • No NS record support (also has limited records types available, compared to Cloudflare)

Overall cons

  • Moving CI over could be a hassle, however using DNSControl could make it easier. (used with Open Domains and Free Domains)
  • URL records, not sure how we would manage to implement these, but we could figure out a way.

wdhdev avatar May 27 '24 07:05 wdhdev

I can write an URL redirector in Python (however would not be a preferable solution, the more preferable one would be you writing one on JS or see the next line) however I think you can also deprecare URL records (maybe send an email to people with URL records to tell them to remove before X?)

CF rate limit: "Record limit, although can be increased by contacting support I believe"

the number of records we have now has already surpassed the limit of 1000 for free and 3500 for paid.

MaskDuck avatar May 27 '24 12:05 MaskDuck

I can write an URL redirector in Python (however would not be a preferable solution, the more preferable one would be you writing one on JS or see the next line) however I think you can also deprecare URL records (maybe send an email to people with URL records to tell them to remove before X?)

We can do this, and just setup a VPS for it to run on.

the number of records we have now has already surpassed the limit of 1000 for free and 3500 for paid.

Cloudflare support can help with this. JS.ORG exceeds that limit I'm pretty sure, and they are on the free plan iirc.

wdhdev avatar May 27 '24 12:05 wdhdev

but we have to reach out for support before the migration

MaskDuck avatar May 27 '24 12:05 MaskDuck

Yep, that's true.

wdhdev avatar May 27 '24 12:05 wdhdev

image

MaskDuck avatar May 27 '24 13:05 MaskDuck

I found something called deSec, which you can also self-host, and haven´t found a limit. https://github.com/desec-io/desec-stack https://desec.io/

zvdxcsite avatar May 27 '24 14:05 zvdxcsite

@zvdxcsite If we did self host our DNS, it would most likely be with a more advanced and more well-known project.

wdhdev avatar May 28 '24 01:05 wdhdev

Also it should be a really good one and good server cause it would probably crash.

NullyIsHere avatar Jun 03 '24 13:06 NullyIsHere

This issue has been marked as stale due to inactivity and will be closed. Comment anything on this issue to prevent it

github-actions[bot] avatar Jun 10 '24 14:06 github-actions[bot]

@phenax After some research (and testing), I think if we did self host our DNS it would be quite beneficial, the only thing is we would need to get at least 2 VPS' with 100% uptime (in 2 different locations, maybe one in the US and another in EU?). Another thing we would need is API support.

After testing Technitium DNS Server, I've found it would probably would work very well. I have personally tried running this and the DNS propagation is extremely fast, and the software itself supports many different record types (including NS) and is super easy to use and understand. With little to no experience in hosting DNS servers, I managed to quite easily host Technitium and setup a DNS zone and got it all up in running with-in <30 minutes. It also has a HTTP API, which would be perfect for our project.

The only main downsides to this is basically learning the API, although I wouldn't think it would be too hard and possibly DNS resolution times would be a bit slower than using a provider like Namecheap or Cloudflare. The software requirements are extremely low, so cost would be very low to run (possibly cheaper than CPanel?). As we use Namecheap as our domain registrar, registering nameservers is very easy.

The reason for 2 VPS' would be one root nameserver and then a secondary DNS server so in the event the root nameserver goes down, the secondary nameserver can still resolve the DNS zone.

ScreenShot1

wdhdev avatar Jun 11 '24 11:06 wdhdev

Seems cool, @wdhdev since I read that i started to work on something for URL records, its almost finish.

NullyIsHere avatar Jun 11 '24 11:06 NullyIsHere

@NullyIsHere Sounds good, let me know when you're finished!

wdhdev avatar Jun 11 '24 11:06 wdhdev

Status update: Working on a script to detect URL records, where and in what domain.

NullyIsHere avatar Jun 16 '24 19:06 NullyIsHere

A few points to consider here:

  • URL redirection is a separate problem to DNS so it can be dealt with separately. So instead of trying to migrate everything at once, we can simplify things a lot by solving 1 problem at a time.
  • It should have preferably no (or at least very high) limits on the number of dns records (and redirections) or rate-limits on the api to manipulate those records.
  • Github should not be the source of truth for data for either of them. I.e. no fetching records directly from github. The records should be saved somewhere else in a more efficient/indexed form.
  • Prefer fixed pricing for our solutions. Since it's more predictable.
  • It should support at least all of our current features.
  • Scaling is a pretty messy problem and I have no idea when scaling becomes a concern for dns servers. If we're going with a self-hosted solution, we'll need to perform extensive load testing and we'll need really good monitoring tools.
  • If we're relying on a managed service, customer service should be really good or else we're stuck dealing with problems we have no control over.
  • Security will be hard to validate but we'll need to keep an eye on it and be as conservative about access as possible.

@wdhdev, Technitium looks great. Appreciate the thorough research! I'll experiment with it a bit this week. If that doesn't work out bunny.net seems like a decent managed alternative (zone management, not the scriptable dns). The pricing is not fixed unfortunately but it seems reasonable enough.

No NS record support (also has limited records types available, compared to Cloudflare)

It seems like bunny supports it here but I'm not sure if there are any implicit restrictions on it.

phenax avatar Jun 17 '24 07:06 phenax

@phenax I've done some testing with Technitium and so far, it seems very good. It even allows you to host your own local domains and TLDs if wanted. It has easy to setup secondary nameservers as well (only a few clicks to setup). It also has in-depth statistics.

Thanks for your response!

wdhdev avatar Jun 17 '24 07:06 wdhdev

also uhm... 20 million queries a month for bunny... in my opinion we will need to like uhm get some stats about the queries: currently are we exceeding 20m queries a month? I doubt that but we need to make sure about that if we're moving to bunny.

MaskDuck avatar Jun 17 '24 08:06 MaskDuck

@MaskDuck @phenax For hosting, uptime and reliability this is what I personally think we would/should do:

  • Main server (ns1.is-a.dev)
    • Hosted on a VPS provider like OVHcloud in the United States
  • Secondary server (ns2.is-a.dev)
    • Hosted on the same provider, but somewhere else in the EU, like Germany or the UK.
  • Possible 3rd server (ns3.is-a.dev) (optional)
    • Also hosted on the same provider, but in Australia.
  • And so on...

This way, if the main nameserver, ns1.is-a.dev goes down, ns2.is-a.dev and ns3.is-a.dev (if it exists), will just pickup the load immediately causing no downtime for DNS queries.

TL;DR; Have 2 or 3 servers in different regions to minimise loading times for users in different regions and to avoid downtime if a server goes down, therefore increasing reliability.

The reason why I recommended OVHcloud is because they have a high SLA for most services and notice in advance for maintenance.

wdhdev avatar Jun 17 '24 08:06 wdhdev

I'd recommend you two host two different nameservers on two different platforms

MaskDuck avatar Jun 17 '24 08:06 MaskDuck

I'd recommend you two host two different nameservers on two different platforms

Same provider, different datacenters will be good enough for our needs, in my opinion. Also, 2 nameservers should be enough, the main one in the US, a secondary one in the EU.

wdhdev avatar Jun 17 '24 08:06 wdhdev

Germany could be a good option as it could cover parts of the East

NullyIsHere avatar Jun 17 '24 09:06 NullyIsHere

Germany could be a good option as it could cover parts of the East

Yeah, I was thinking Germany.

wdhdev avatar Jun 17 '24 11:06 wdhdev

Germany could be a good option as it could cover parts of the East

Yeah, I was thinking Germany.

The server that used to run hosting and manage site is Located in London, England that could also become part of this as it only running the discord bot and open-domains plus it has a 25GB uplink

andrewstech avatar Jun 17 '24 12:06 andrewstech

The server that used to run hosting and manage site is Located in London, England that could also become part of this as it only running the discord bot and open-domains plus it has a 25GB uplink

We will need DDoS protection & stuff. Unfortunately, we probably won't be able to use your server unless Phenax wants to of course.

wdhdev avatar Jun 17 '24 23:06 wdhdev

I might be wrong but from what I can tell, Technitium only works on disk. Which means managing multiple instances with that will be a shit show. Both on how we update dns records across instances and how we scale instances.

We'll need a common data source that lives outside disk. This is because killing an instance should not mean all the records are gone and we now have the headache of migrating the records on disk to the new instance before that. Which we may not be able to do reliably. So we'll need 2 instances running the dns server and 1 db instance with backups.

If the caching strategy isn't good enough, a ddos attack could potentially overload the db. But the chance of that is pretty slim and can be dealt with later with replication. We'll have a better idea after some stress testing and tweaking our dns solution a bit.

PowerDNS seems like a pretty strong alternative that supports external backends for storage and has decent caching which means we can deploy multiple instances of this and scale it without much hassle. Also has an api and a dashboard. Will try this out later.

Let me know what you all think.

phenax avatar Jun 18 '24 08:06 phenax

@phenax For Technitium, you just setup 2 or more instances, run the zone on, ns1.is-a.dev, then configure secondary zones on the other nameservers, which point to ns1.is-a.dev, after adding NS records for each secondary nameserver in the primary zone. The zone is then automatically synced to secondary servers.

I have my own setup using Technitium including a secondary nameserver and I'm happy to show you how I have it configured. Me and @MaskDuck were just doing some testing with it a little while ago. I can hop in a Discord call with you if you want, anytime today, just DM me or ping me in the support server.

I have seen PowerDNS before, however it seems a bit complicated to setup, at least for me.

Also, in my opinion, it is better to use a disk backend. The reason for this is because that way if the main Technitium instance goes down (or if using PowerDNS, the external backend), it will not disrupt traffic at all, as the zone is striped across all secondary nameservers.

wdhdev avatar Jun 18 '24 09:06 wdhdev

@phenax I've sent you a message on Discord showing you how to manage Technitium with multiple instances and some recommended options to enable, with screenshots. It is super easy to setup and to scale. The main downside is just having a login for each instance, other than that, it is super easy to scale.

wdhdev avatar Jun 18 '24 10:06 wdhdev

The server that used to run hosting and manage site is Located in London, England that could also become part of this as it only running the discord bot and open-domains plus it has a 25GB uplink

We will need DDoS protection & stuff. Unfortunately, we probably won't be able to use your server unless Phenax wants to of course.

My server is now equipped with DDOS protection, Upgraded my plan last month. Also all servers must have an IPV6 address otherwise AAAA records won’t be resolved

andrewstech avatar Jun 18 '24 11:06 andrewstech

My server is now equipped with DDOS protection, Upgraded my plan last month.

Alright then, we can most likely configure it as a secondary nameserver, the more nameservers the better 😆

I might consider hosting one as well.

Also all servers must have an IPV6 address otherwise AAAA records won’t be resolved

I've just tested my Technitium instance, which does not have an IPv6 assigned and AAAA records do in fact resolve. You can see the output here:

$ dig ipv6.test.wdh.gg AAAA +noall +answer
ipv6.test.wdh.gg.       3600    IN      AAAA    2001:4860:4860::8888

image

wdhdev avatar Jun 18 '24 12:06 wdhdev

One problem i thought about is that ns cant be n1.is-a.dev as to resolve that needs to resolve n1.is-a.dev, etc. So it should be n1.wdh.gg oor similar

NullyIsHere avatar Jun 18 '24 12:06 NullyIsHere

One problem i thought about is that ns cant be n1.is-a.dev as to resolve that needs to resolve n1.is-a.dev, etc. So it should be n1.wdh.gg oor similar

We can use glue records to resolve this. They are configured on the domain before the nameserver. They sort of sit on top

https://www.ibm.com/blog/understanding-glue-records-and-dedicated-dns/

andrewstech avatar Jun 18 '24 12:06 andrewstech