domain_exporter
domain_exporter copied to clipboard
caarlos0
could not parse whois response from .de tld
Expiration date parsing error
ERRO[0497] failed to probe "my_domain": could not parse date: -date: 2021-09-20 source="domain.go:65"
.de TLD does not return any expiration times on the domains I tested at least... not much we can do...
So. In internal/whois/whois.go
if you have returned \t in expiration date/time your domain like i had, you should separate or remove "\t" to work fine your code, and if you put that code
dateStr = after(dateStr, "-date:\t") right after line "71" It will works fine.
And after that i'm trying to build a container of that, and i got error says can not copy apk to /tmp/ even i build with your dockerfile.
Thanks for reply.
also go test -v ./... will test it (and you can add a case specific for this problem as well)
.de TLD does not return any expiration times on the domains I tested at least... not much we can do...
Some register can explain expiration date, many possible add it for default check?
namecheap & godaddy
example: whois -h whois.godaddy.com adspro.eu Domain Name: adspro.eu Registry Domain ID: adspro_eu-EURID Registrar WHOIS Server: whois.godaddy.com Registrar URL: http://www.godaddy.com Updated Date: 2020-08-20T13:06:45Z Creation Date: 2018-01-30T01:16:22Z Registrar Registration Expiration Date: 2022-01-30T22:59:59Z
GoDaddy
All the help and tools you need to grow online: Websites, Domains, Digital and Social Marketing - plus GoDaddy Guides with you every step of the way
GoDaddy
same for certain extensions ie. .vn, returning value -1 nice if there's a config to customize whois server based on extension :) for example .de$ whois.godaddy.com
Probably need to improve this RE https://github.com/caarlos0/domain_exporter/blob/master/internal/whois/whois.go#L52
so this happens: https://github.com/caarlos0/domain_exporter/blob/master/internal/whois/whois.go#L112-L115
GitHub
Exports the expiration time of your domains as prometheus metrics. - caarlos0/domain_exporter
Exports the expiration time of your domains as prometheus metrics. - caarlos0/domain_exporter
I have the same issue with .kz domains Maybe adding specific whois servers to some domain zones will be a good idea.
It's not really possible to do overrides per TLD. For example, if you use whois.godaddy.com to query for europa.eu you'll get no match since GoDaddy is not its registrar (it's Proximus/Belgacom, which doesn't run a whois server so you're limited to the output EURid provides).
The same goes for other TLDs. For example the GoDaddy whois server won't respond for spiegel.de. Their registrar is cloudns, which doesn't seem to have a functional whois server either so you're limited to DENICs response which doesn't include expiration/renewal info.
In most cases, whois can determine through the IANA database if there's a more specific server that can answer the query, and will return it as a refer if you query IANA. However, these point to the whois servers of TLD registry operators, and not domain registrars (which makes sense since the latter can change often and dynamically).
$ whois -h whois.iana.org adspro.eu
% IANA WHOIS server
% for more information on IANA, visit http://www.iana.org
% This query returned 1 object
refer: whois.eu
Most of these whois servers don't return much past the creation/registration of the domain or the last modification of the domain (which doesn't tell you much).
Using whois works pretty well for .net/.com/.org, but it's going to be very hit and miss for other TLDs. You'll need the ability to override the whois server used by registered domain, and hope that the registrar in question runs a whois server and that their whois server happens to expose this piece of data.
That would do the trick. Folks will just have to ensure they pick a TLD or registrar that makes that information available this way. Otherwise you'd have to integrate with registrar APIs directly.
I am getting -1 as value for .in extensions. Can anyone please help me with this? Thanks in advance.
Hello. Error parsing "ch" and "es" domains
11:29AM ERR failed to probe ***.ch error="could not parse whois response: \"Requests of this client are not permitted. Please use https://www.nic.ch/whois/ for queries.\\n\\n\""
ERR failed to probe ***.es error="could not parse whois response: \"\\r\\n Conditions of use for the whois service via port 43 for .es domains\\r\\n\\r\\n Access will only be enabled for IP addresses authorised by Red.es. A maximum of one IP address per\\r\\n user/organisation is permitted.\\r\\n\\r\\n Red.es accepts no responsibility whatsoever for the availability of access to WHOIS, which may be\\r\\n suspended at any time and without prior warning at the discretion of the public entity.\\r\\n\\r\\n The service will be limited to the data established by Red.es.\\r\\n\\r\\n The user promises to make use of the service and to carry out any action derived from the aforesaid\\r\\n use in accordance with current applicable regulations, in particular with legislation on “.es” domain\\r\\n names and personal data protection.\\r\\n\\r\\n In particular, the user undertakes not to use the service to carry out abusive or speculative domain\\r\\n name registrations, pursuant to section 5 of the Sixth Additional Provision of Law 34/2002, of 11 July,\\r\\n on Services of the Information Society and Electronic Commerce. Likewise, the User undertakes not to\\r\\n use the service to obtain data, the possession of which may contravene the provisions of Organic Law\\r\\n 15/1999, of 13 December, on Personal Data Protection, and its Regulations, or in Law 34/2002, of 11\\r\\n July, on Services of the Information Society and Electronic Commerce.\\r\\n\\r\\n Failure to comply with these conditions will result in the immediate withdrawal of the service and any\\r\\n registered domain name which breaches said conditions may be officially cancelled by Red.es.\\r\\n -------------------------------------------------------------------------------------------------------\\r\\r\\n\\r\\n The IP address used to perform the query is not authorised or has exceeded the established limit for\\r\\n queries.To request access to the service,complete the form located at https://sede.red.gob.es/sede/whois,\\r\\n where you may also consult the service conditions.\\r\\n\\r\\n -------------------------------------------------------------------------------------------------------\\r\\n More information on each domain may be consulted at www.dominios.es.\\r\\n\\x00\""
By default for *.ch, you'll hit nic.ch, but they don't allow requests using the WHOIS protocol.
$ whois nic.ch
Requests of this client are not permitted. Please use https://www.nic.ch/whois/ for queries.
This won't work for any .ch domain by default, unless the registrar of the particular .ch domain you're interested in runs a WHOIS server you can use instead.
For Spanish domains, you'll have the same issue. WHOIS is no longer available for the TLD, unless you the registrar still runs one:
$ whois thelocal.es
This TLD has no whois server, but you can access the whois database at
https://www.nic.es/
You can get slightly more info with:
$ whois -h whois.nic.es thelocal.es
Conditions of use for the whois service via port 43 for .es domains
Access will only be enabled for IP addresses authorised by Red.es. A maximum of one IP address per
user/organisation is permitted.
Red.es accepts no responsibility whatsoever for the availability of access to WHOIS, which may be
suspended at any time and without prior warning at the discretion of the public entity.
The service will be limited to the data established by Red.es.
The user promises to make use of the service and to carry out any action derived from the aforesaid
use in accordance with current applicable regulations, in particular with legislation on “.es” domain
names and personal data protection.
In particular, the user undertakes not to use the service to carry out abusive or speculative domain
name registrations, pursuant to section 5 of the Sixth Additional Provision of Law 34/2002, of 11 July,
on Services of the Information Society and Electronic Commerce. Likewise, the User undertakes not to
use the service to obtain data, the possession of which may contravene the provisions of Organic Law
15/1999, of 13 December, on Personal Data Protection, and its Regulations, or in Law 34/2002, of 11
July, on Services of the Information Society and Electronic Commerce.
Failure to comply with these conditions will result in the immediate withdrawal of the service and any
registered domain name which breaches said conditions may be officially cancelled by Red.es.
-------------------------------------------------------------------------------------------------------
The IP address used to perform the query is not authorised or has exceeded the established limit for
queries.To request access to the service,complete the form located at https://sede.red.gob.es/sede/whois,
where you may also consult the service conditions.
-------------------------------------------------------------------------------------------------------
More information on each domain may be consulted at www.dominios.es.
Basically, you need to get whitelisted.
Fwiw, Switch (which operates .ch and .li) has an RDAP server:
- https://rdap.nic.ch/domain/apple.ch
- https://rdap.nic.ch/domain/apple.li
For some reason it isn't listed in the IANA bootstrap data — https://data.iana.org/rdap/dns.json — but it's here in a different project's overrides file:
https://github.com/meeb/whoisit/blob/main/whoisit/overrides.py
we are having the same problem with .es TLD.
we also noticed if we use godaddy whois servers (whois.godaddy.com) we are able to get a proper response:
Click to see cli output
output
➜ ~ whois -h whois.godaddy.com $REDACTED_DOMAIN_NAME
Domain Name: $REDACTED_DOMAIN_NAME
Registry Domain ID: 13669014911570_DOMAIN-KEYSYS
Registrar WHOIS Server: whois.godaddy.com
Registrar URL: https://www.godaddy.com
Updated Date: 2023-01-22T11:12:14Z
Creation Date: 2012-02-17T14:38:00Z
Registrar Registration Expiration Date: 2024-02-17T12:12:14Z
Registrar: GoDaddy.com, LLC
Registrar IANA ID: 146
Registrar Abuse Contact Email: [email protected]
Registrar Abuse Contact Phone: +1.4806242505
Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited
Domain Status: clientRenewProhibited https://icann.org/epp#clientRenewProhibited
Domain Status: clientDeleteProhibited https://icann.org/epp#clientDeleteProhibited
Registrant Organization:
Registrant State/Province: NONE
Registrant Country: XX
Registrant Email: Select Contact Domain Holder link at https://www.godaddy.com/whois/results.aspx?domain=$REDACTED_DOMAIN_NAME
Admin Email: Select Contact Domain Holder link at https://www.godaddy.com/whois/results.aspx?domain=$REDACTED_DOMAIN_NAME
Tech Email: Select Contact Domain Holder link at https://www.godaddy.com/whois/results.aspx?domain=$REDACTED_DOMAIN_NAME
Name Server: $REDACTED_NS
Name Server: $REDACTED_NS
Name Server: $REDACTED_NS
Name Server: $REDACTED_NS
DNSSEC: unsigned
URL of the ICANN WHOIS Data Problem Reporting System: http://wdprs.internic.net/
>>> Last update of WHOIS database: 2023-03-01T13:19:11Z <<<
➜ ~ whois $REDACTED_DOMAIN_NAME
% IANA WHOIS server
% for more information on IANA, visit http://www.iana.org
% This query returned 1 object
refer: whois.nic.es
domain: ES
organisation: Red.es
address: Edificio Bronce
address: Plaza Manuel Gomez Moreno
address: Madrid 28020
address: Spain
contact: administrative
name: Alberto Martinez Lacambra
organisation: Red.es
address: Edificio Bronce
address: Plaza Manuel Gomez Moreno
address: Madrid 28020
address: Spain
phone: +34 91 212 76 24
fax-no: +34 91 555 76 64
e-mail: [email protected]
contact: technical
name: Juan Antonio Gutierrez Gil
organisation: Red.es
address: Edificio Bronce
address: Plaza Manuel Gomez Moreno
address: Madrid 28020
address: Spain
phone: +34 91 212 76 20
fax-no: + 34 91 556 88 64
e-mail: [email protected]
nserver: A.NIC.ES 194.69.254.1 2001:67c:21cc:2000:0:0:64:41
nserver: C.NIC.ES 194.0.34.53 2001:678:44:0:0:0:0:53
nserver: G.NIC.ES 2001:500:14:7001:ad:0:0:1 204.61.217.1
nserver: H.NIC.ES 194.0.33.53 2001:678:40:0:0:0:0:53
ds-rdata: 44375 8 2 9d9858ae981aa53dd1143d93844e3d69b0fb73a9b4fe5759da39e036e754d402
whois: whois.nic.es
status: ACTIVE
remarks: Registration information: http://www.nic.es/
created: 1988-04-14
changed: 2022-12-19
source: IANA
# whois.nic.es
Conditions of use for the whois service via port 43 for .es domains
Access will only be enabled for IP addresses authorised by Red.es. A maximum of one IP address per
user/organisation is permitted.
Red.es accepts no responsibility whatsoever for the availability of access to WHOIS, which may be
suspended at any time and without prior warning at the discretion of the public entity.
The service will be limited to the data established by Red.es.
The user promises to make use of the service and to carry out any action derived from the aforesaid
use in accordance with current applicable regulations, in particular with legislation on “.es” domain
names and personal data protection.
In particular, the user undertakes not to use the service to carry out abusive or speculative domain
name registrations, pursuant to section 5 of the Sixth Additional Provision of Law 34/2002, of 11 July,
on Services of the Information Society and Electronic Commerce. Likewise, the User undertakes not to
use the service to obtain data, the possession of which may contravene the provisions of Organic Law
15/1999, of 13 December, on Personal Data Protection, and its Regulations, or in Law 34/2002, of 11
July, on Services of the Information Society and Electronic Commerce.
Failure to comply with these conditions will result in the immediate withdrawal of the service and any
registered domain name which breaches said conditions may be officially cancelled by Red.es.
-------------------------------------------------------------------------------------------------------
The IP address used to perform the query is not authorised or has exceeded the established limit for
queries.To request access to the service,complete the form located at https://sede.red.gob.es/sede/whois,
where you may also consult the service conditions.
-------------------------------------------------------------------------------------------------------
More information on each domain may be consulted at www.dominios.es.
#180 is created as a solution for this
