uptime-kuma
uptime-kuma copied to clipboard
louislam
Domain Expiry Monitor
⚠️ Please verify that this feature request has NOT been suggested before.
- [X] I checked and didn't find similar feature request
🏷️ Feature Request Type
New Monitor
🔖 Feature description
It would be awesome if we can add domains, and monitor their expiry status.
✔️ Solution
We should be notified when there is a change to the DNS or expiry date, and be notified x days before expiry.
❓ Alternatives
No response
📝 Additional Context
11Uptime has a similar feature, as well as Better Uptime
I think it is hard to retrieve domain information from tld companies/organization. For example there is no public api for hungarian domains.
If you want to detect dns (host->ip) resolve changes that is other issue.
it is hard to retrieve domain information from tld companies/organization
It is hard and not reliable without a standardized API, but for many domains WHOIS scraping would go a long way. Still not too fun to implement...
Seems like there are quite a few APIs, some with big limits, and some completely free: https://blog.api.rakuten.net/top-domain-whois-api/
Just found a library. I think it is quite promising, because it directly queries the whois 43 port server. As long as Uptime Kuma caches the expiration date, it should not hit the rate limit. https://github.com/eithan1231/whois-light
+1 for this. I was looking for something self-hosted for this when I came across domain-monitor.io, and that looked suspiciously like Uptime Kuma so I came over here to search. This and SSL expiry notifications would be super cool.
Just found a library. I think it is quite promising, because it directly queries the whois 43 port server. As long as Uptime Kuma caches the expiration date, it should not hit the rate limit. https://github.com/eithan1231/whois-light
According to open issues, it seems that the library is not completely consistent for all TLDs. This could have a negative effect on the implementation into Uptime Kuma, as it would not be possible to have one code for all TLDs.
If you are interested, I could try to look at it and try to make a universal WHOIS library for most domains. But I can't promise anything.
Would that help?
@louislam As I wrote in the comment above, I started working on a library to retrieve domain data from WHOIS servers.
I'm posting a link to my GitHub repository.
You're overcomplicating it, just let users enter an expiration date for the domain name when adding a new monitor. And then send notifications when the specified date approaches.
+1 for this. I'm commenting in support because I can't tell where things landed with the request, though I did see the Github repo and closed, duplicate request.
Regarding the comment before mine, I have to say it completely misses the point: the reason for the feature request is to account for absentmindedness/human error and if we all took the time to track when domains were due for renewal then there would be no need for such a thing... but people are going to people. I love the idea of Uptime Kuma offering an additional check, and I already have every domain set to autorenew. Sometimes things happen and people forget to update their credit card's expiration date for instance.
@anseltaft In fact, a big problem is the inconsistent WHOIS servers of different companies. As I wrote above, I started working on the Universal WHOIS library, but basically most registrars had different results when requesting information about the domain. Personally, I think that we are currently unable to provide full support for most domain extensions, and therefore this requirement is in a frozen state.
I'm not sure if I once posted recent changes to GH, but if you look at my repository, you'll probably understand what I'm talking about.
@anseltaft In fact, a big problem is the inconsistent WHOIS servers of different companies. As I wrote above, I started working on the Universal WHOIS library, but basically most registrars had different results when requesting information about the domain. Personally, I think that we are currently unable to provide full support for most domain extensions, and therefore this requirement is in a frozen state.
I'm not sure if I once posted recent changes to GH, but if you look at my repository, you'll probably understand what I'm talking about.
That is why above I suggested the possibility of manually adding a domain expiration date, it is better than nothing and better than constant breakdowns whois, but I was downvoted
@louislam would you be open to the idea of using NLP to tackle this issue? For granted, it will add some heavy dependencies to the project, but it will allow us to cover all WHOIS providers.
@alexandernst In fact, I thought about it too. I planned to try and eventually use it in my library, and it could then be used as the only dependency. But I haven't gotten to it yet... maybe on the weekend. However, it is true that at the moment this is probably the only solution.
Would love to have this feature, even if it's something manually entered somewhere for now (Having "Check domain expiry for xxx.tld" show up in slack at 30, 14, and 7 days before would be just fine to let the team know to check in this, and update the date in Uptime Kuma if needed).
On a side note, I did discover this check domain expiry plugin for Nagios, and it's not trivial (it's written in SH, and uses the whois tool directly). Search for the get_expiration() function.
https://github.com/glensc/monitoring-plugin-check_domain/blob/master/check_domain.sh
RDAP is the replacement for WHOIS, see some more details on Wikipedia. It's very useful as it implements a structured JSON format for representing domain metadata, including the expiration.
The way it'd work is that Uptime-Kuma would have to cache the RDAP Bootstrap list (https://data.iana.org/rdap/dns.json) to get a list of TLD -> RDAP servers and then query the RDAP server on a regular basis for the expiry date.
RDAPis the replacement for WHOIS, see some more details on Wikipedia. It's very useful as it implements a structured JSON format for representing domain metadata, including the expiration.The way it'd work is that Uptime-Kuma would have to cache the RDAP Bootstrap list (
https://data.iana.org/rdap/dns.json) to get a list of TLD -> RDAP servers and then query the RDAP server on a regular basis for the expiry date.
Wow, it could be so helpful.
Surprisingly, I don't know why I didn't find anything about RDAP.
Haven't tested it yet, but I just found a nodejs package for rdap: https://github.com/dcoffey-zengenti/node-rdap
Looks like not all TLDs have working RDAP yet (.io seems to be one, see: https://openrdap.org/demo?cmd=rdap+-v+codepen.io vs https://www.whois.com/whois/codepen.io ), but this would be better than nothing and RDAP should be the path for this check going forward.
Looks like not all TLDs have working RDAP yet, but this would be better than nothing and RDAP should be the path for this check going forward.
Yeah, all the generic TLDs must implement RDAP per ICANN policies, but the country codes are free to do whatever their government wants.
I hope this gains some tractions and seems like it would be trivial to add for the most part. Yes some of the TLDs aren't doing RDAP, in those cases, a simple whois seems to work just fine. I get it, there is no standardization around whois and what is returned....But I think RDAP would cover a large chunk of domains out there.
You are right, the output of WHOIS is always different, except for the line:
Registry Expiry Date:
Although, just as mentioned above, I also did not find these details in the RFC I haven't seen output for any existing domain name that doesn't have an expiration line, please correct me if the expiration output is different somewhere.
I have a Remote Probe implementation to work with Push monitor type that has a function to check for Domain Expiration. May help somebody.
https://github.com/zimbres/UptimeKumaRemoteProbe
Looking at some stats about RDAP support (source https://deployment.rdap.org/) it looks like 72.1% of domains support it, with all gTLDs supporting it but only 22.7% of ccTLDs supporting. Given that 72.1% of domains support it, I would be inclined to suggest that we start by implementing a domain expiry using RDAP only. That should simplify implementation a bit hopefully and still cover the majority of use cases. If in future it is determined that accessing those domains that don't currently support RDAP, we could perhaps think about how we want to deal with that problem (maybe if enough people ask ccTLDs for RDAP, they will provide? :wink:).
I think that I might have a bit of a play with this over the coming days and see if I can get anything working. As far as implementation, would we want to create an entirely new monitor type (something with a long interval), or something more like what we have with the certificate expiry, with the monitors URL / hostname being used as the domain to check?
I know about #1775 but that make use of whois which we know to be a bit problematic and it hasn't been updated for over a year.
I would argue that this should be a dedicated monitor, but this is just
- my feeling of how generally useful (as in useful for everybody),
- technical debt (how complex is the monitor?) and
- UX-friedly (there are plenty of options for monitors already ⇒ adding more (without them being required) might be overwhelming)
If you decide that this is a good decision, please have a look at the new way of writing monitors (#3178) instead of the one used in older PRs ^^
whoiser package works great. The initial check should occur when adding a site and subsequent checks should be carried out periodically, similar to how SSL certificate checks are done.
+1
I have several domain names where I would want to get a 90 day advance expiration (or WHOIS data change notification). This to allow me to find the cheapest domain renewal rates and so on.
@TheLinuxGuy We use 👍🏻 on issues to prioritise work, as always: Pull Requests welcome.
An example-pr that implements a monitor according to our new syntax (not just piling anything into monitor.js) can be found here: https://github.com/louislam/uptime-kuma/pull/3178 or https://github.com/louislam/uptime-kuma/pull/3308