server icon indicating copy to clipboard operation
server copied to clipboard

[Bug]: App Updates with IPv6 and Debian Trixie: RequestException cURL error 60: SSL: no alternative certificate subject name matches target hostname 'github.com'

Open gohrner opened this issue 1 month ago • 11 comments

⚠️ This issue respects the following points: ⚠️

Bug description

On several servers, since the upgrade from Debian Bookworm (12) to Trixie (13) I get the following error whenever I try to update Nextcloud apps:

RequestException
cURL error 60: SSL: no alternative certificate subject name matches target hostname 'github.com' (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://github.com/nextcloud-releases/contacts/releases/download/v8.0.6/contacts-v8.0.6.tar.gz

This issue has already been experienced several times in the past:

  • https://help.nextcloud.com/t/updating-installing-apps-always-choses-ipv6-but-github-com-hasnt-an-aaaa-entry/211276
  • https://help.nextcloud.com/t/requestexception-curl-error-60-ssl-no-alternative-certificate-subject-name-matches-target-hostname-github-com/208178

It still does not seem to be fixed, and I could not find any official bug report for it.

This probably also happens on other OSes, not only Debian Trixie, but I didn't experience this issue with Debian Bookworm.

Steps to reproduce

  1. Install Nextcloud using the tarball on an IPv6 enabled Debian Trixie server. The server's DNS server probably needs to have IPv6 wildcard subdomain resolution to some IPv6 address where an SSL supporting web server must listen.
  2. Try to update installed apps which have an update available using the Web UI.
  3. The update fails with the abovementioned message - of course, the domain names may vary, depending on the package.

Expected behavior

Updating Apps is possible if an update is available.

Nextcloud Server version

32

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.4

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 31 to 32)

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • [x] Default user-backend (database)
  • [x] LDAP/ Active Directory
  • [ ] SSO - SAML
  • [ ] Other

Configuration report


List of activated Apps


Nextcloud Signing status


Nextcloud Logs


Additional info

No response

gohrner avatar Nov 16 '25 19:11 gohrner

To work around the issue, I used the approach stated in https://help.nextcloud.com/t/updating-installing-apps-always-choses-ipv6-but-github-com-hasnt-an-aaaa-entry/211276

This is definitely not a proper solution, but for now allows me to update Nextcloud Apps again.

--- 3rdparty/guzzlehttp/guzzle/src/Handler/StreamHandler.php.orig       2025-10-27 18:15:28.447846943 +0100
+++ 3rdparty/guzzlehttp/guzzle/src/Handler/StreamHandler.php    2025-11-16 21:57:02.713299977 +0100
@@ -357,6 +357,8 @@
     {
         $uri = $request->getUri();
 
+       if (empty($options['force_ip_resolve'])) $options['force_ip_resolve'] = 'v4';
+
         if (isset($options['force_ip_resolve']) && !\filter_var($uri->getHost(), \FILTER_VALIDATE_IP)) {
             if ('v4' === $options['force_ip_resolve']) {
                 $records = \dns_get_record($uri->getHost(), \DNS_A);
--- 3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php.orig 2025-10-27 18:15:28.447846943 +0100
+++ 3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php      2025-11-16 21:57:38.289778029 +0100
@@ -506,6 +506,8 @@
             $conf[\CURLOPT_TIMEOUT_MS] = $options['timeout'] * 1000;
         }
 
+       if (empty($options['force_ip_resolve'])) $options['force_ip_resolve'] = 'v4';
+
         // CURL default value is CURL_IPRESOLVE_WHATEVER
         if (isset($options['force_ip_resolve'])) {
             if ('v4' === $options['force_ip_resolve']) {

gohrner avatar Nov 16 '25 21:11 gohrner

The server's DNS server probably needs to have IPv6 wildcard subdomain resolution to some IPv6 address where an SSL supporting web server must listen.

Indeed. This is the root cause.

Your DNS server/setup needs to be fixed. This isn't a Nextcloud bug unless I'm misunderstanding something.

This DNS behavior will break all sorts of things in an IPv6-only environment (at least when attempting to connect to any IPv4-only resources). I don't think NAT64/DNS64 is even possible under these circumstances.

It makes sense that forcing IPv4 only resolution fixes this, but that's only a hack.

joshtrichards avatar Nov 17 '25 20:11 joshtrichards

@joshtrichards: curl on command line, and wget and similar work just fine with the URLs listed in the exception messages.

gohrner avatar Nov 17 '25 20:11 gohrner

Are those command line tests with curl ultimately using a NAT64/DNS64 address to connect to GitHub? If so they're somehow not being provided with whatever that other wildcard AAAA record is that your PHP environment is getting.

Unfortunately there can be differences in how DNS resolution is handled even on the same system.

For example, curl may use c-ares (depending on how it's compiled), while PHP I believe generally uses the system library (libc).

joshtrichards avatar Nov 17 '25 21:11 joshtrichards

An additional thought: there could be an interaction with NAT64 unless Nextcloud's allow_local_remote_address is set to true (it's not by default).

I wouldn't expect quite the failure mode you're seeing, but I'd need to go back and review those code paths again to be sure.

joshtrichards avatar Nov 17 '25 22:11 joshtrichards

@joshtrichards: I haven't tried enabling allow_local_remote_address yet, as I haven't researched what its additional implications are, but in one of posts / forum threads linked above someone wrote that it helped for him:

  • https://help.nextcloud.com/t/requestexception-curl-error-60-ssl-no-alternative-certificate-subject-name-matches-target-hostname-github-com/208178/7

I'd have expected the name resolution to fall back to the A record first if no AAAA record exists, instead of first trying to construct additional FQDNs by appending the search suffix - though I probably have a flawed understanding about the name resolution priorities in conjunction with IPv4, IPv6 and dual stack machines.

I'm not sure though why NAT64 should come into play, as the server is no IPv6-only-host, but dual stack (otherwise the patch above would not have been usable, I think).

curl on command line says:

# curl -v https://github.com/nextcloud-releases/contacts/releases/download/v8.0.6/contacts-v8.0.6.tar.gz 
* Host github.com:443 was resolved.
* IPv6: (none)
* IPv4: 140.82.121.4
*   Trying 140.82.121.4:443...
(...)

gohrner avatar Nov 19 '25 10:11 gohrner

Hey, i also faced this issue recently on one of my Nextcloud Server. The resolution was, that i needed to remove the search entry from the /etc/resolv.conf of the system. (lets assume the search domain was example.org)

I also had the behavior, that, after the AAAA query for github.com returned an NXDOMAIN, the system tried to resolve the AAAA record of github.com.example.org, which returned some random IPv6 addresses of the wildcard AAAA record, that was configured for this domain. My expectation was, that the system will try the A record of github.com after the AAAA record of github.com returned NXDOMAIN. The, lets call it "early", fallback to the search domain is quite unexpected for me.

(Note: The wildcard record wasn't pointing to the Nextcloud server in my scenario.)

Currently, i am also searching the reason for this behavior. I can also confirm, that this only happened in the Nextcloud app updater and not with curl or wget (well, not completely true, it actually happened there, but only with the -6 flag).

In my situation, i can confirm that no NAT64/DNS64 is involved.

For example, curl may use c-ares (depending on how it's compiled), while PHP I believe generally uses the system library (libc).

Nice catch, but the Debian version of curl is compiled without c-ares since 2010, as there were serveral problems. I also confirmed that on Debian Trixie and Bookworm systems (curl -V and phpinfo();). php-curl is using actually the same curl / libcurl as one uses with the command line tool curl. And thats a fact, which turns me sceptical: in both situations, the same libcurl was used, which uses the libc implementation for DNS resolving, but it behaves obviously different.


This isn't a Nextcloud bug unless I'm misunderstanding something.

This is an Nextcloud issue. Definitively. Nextcloud is having an own DNS Middleware (https://github.com/nextcloud/server/blob/master/lib/private/Http/Client/DnsPinMiddleware.php), which is basically doing the following in dnsResolve():

  1. Explicitely resolve the A record
  2. Explicitely resolve the AAAA record
  3. Explicitely resolve an CNAME record

By explicitely resolving the AAAA record (via PHP's native dns_get_record), you are getting the same effect as if you invoke curl or wget with -6 - no AAAA record on github.com, so it falls back to github.com.example.org (search domain at the end).

It doesnt really look like that this behavior is quite new or related to any recent code or implementation changes. At least it also happens on Debian Bookworm (as well as Trixie) and also happens with an older PHP version. I mean theres nobody to blame at this point, this is the correct and expected behavior, if you explicitely resolve an AAAA record...

I think that this comes up at the point, where an wildcard record is added to the search domain or the search domain is configured or changed on the system.

One possibility to solve the problem would be, if we check if the actually resolved domain is the domain, that we requested:

                                foreach ($dnsResponses as $dnsResponse) {
                                        if (isset($dnsResponse['host'])) {
                                                if ($dnsResponse['host'] != $target) {
                                                        continue;
                                                }
                                        }

                                        if (isset($dnsResponse['ip'])) {
                                                $targetIps[] = $dnsResponse['ip'];
                                                $canHaveCnameRecord = false;
                                        } elseif (isset($dnsResponse['ipv6'])) {
                                                $targetIps[] = $dnsResponse['ipv6'];
                                                $canHaveCnameRecord = false;
                                        } elseif (isset($dnsResponse['target']) && $canHaveCnameRecord) {
                                                $targetIps = array_merge($targetIps, $this->dnsResolve($dnsResponse['target'], $recursionCount));
                                        }
                                }

At least on my box, this is solving the problem. I am not sure, whether this is the perfect solution and which side effects we may face in some edge cases with that, but its at least an option. I hope, that some Nextcloud Devs will may have a look on that and tell us their opinion.

EDIT: To be clear - my proposed solution is a full knock out according to the system wide configured search domain, it will lead to the fact, that you cannot resolve any DNS records with the search domain in Nextcloud context anymore. But i am also not sure, in which scenario one really wants to do that. Especially regarding to HTTPS i cant really imagine any real world scenario, where anyone would do that.

Alternatively (that would be the correct solution), we need to do the following:

  1. resolve the AAAA record while sorting input - output mismatches out
  2. resolve the A record while sorting input - output mismatches out
  3. if response list is empty: resolve the AAAA record while keeping input - output mismatches
  4. if response list is still empty: resolve the A record while keeping input - output mismatches

but that would be a quite complex code change with a questionable result, at least imho.

@joshtrichards As you seem to come with much more code-knowledge about Nextcloud as I, what are you thinking about this? Or maybe also @LukasReschke as initial code owner of the DnsMiddleware?

WhoAmI0501 avatar Dec 05 '25 20:12 WhoAmI0501

@WhoAmI0501: Thanks your your thorough analysis! You put in quote some effort.

Currently, i am also searching the reason for this behavior. I can also confirm, that this only happened in the Nextcloud app updater and not with curl or wget (well, not completely true, it actually happened there, but only with the -6 flag).

Regarding the -6 argument, the behaviour is what I'd expect - there is no AAAA record, so fall back to using the search domain. That's - at least according to my current understanding - what it's purpose is.

However, if you're not forcing it to use IPv6 only resolution, it first falls back to try the A record, which Nextcloud does not.

Regarding your references to older Debian versions: Not sure if I understood you completely correctly there, however in my case, I didn't have the issue with Debian 12 "Bookworm" and Nextcloud 31.

It only started to appear with Nextcloud 32 and Debian 13 "Trixie" - unfortunately the update of both was so close together that I cannot really say which of both broke it.

gohrner avatar Dec 06 '25 13:12 gohrner

However, if you're not forcing it to use IPv6 only resolution, it first falls back to try the A record, which Nextcloud does not.

Sure, the reason is, that the DnsMiddleware of Nextcloud performs a loop over the record types A, AAAA, and CNAME, and then calls PHP's dns_get_record for each record type explicitely. The thing is, that dns_get_record doesn't support the "normal" fallback behavior, you must specify an record type. And therefore, you also see a similar effect, which you can see, if you e.g. use curl or wget with -6. The DnsMiddleware of Nextcloud doesn't have a proper fallback mechanism, as well as dns_get_record itself. (https://www.php.net/manual/en/function.dns-get-record.php)

PHP is basically using getnameinfo of glibc:

php-src on  HEAD (5a44c76) via 🐘 v8.4.15 
❯ grep -iIrF "getnameinfo"
ext/standard/dns.c:		if (getnameinfo((struct sockaddr *)&sa6, sizeof(sa6), out, sizeof(out), NULL, 0, NI_NAMEREQD) != 0) {
ext/standard/dns.c:		if (getnameinfo((struct sockaddr *)&sa4, sizeof(sa4), out, sizeof(out), NULL, 0, NI_NAMEREQD) != 0) {
ext/standard/net.c:	/* Fallback on getnameinfo() */
ext/standard/net.c:			if (getnameinfo(addr, addrlen, ZSTR_VAL(ret), NI_MAXHOST, NULL, 0, NI_NUMERICHOST) == SUCCESS) {

It only started to appear with Nextcloud 32 and Debian 13 "Trixie" - unfortunately the update of both was so close together that I cannot really say which of both broke it.

On Debian Bookworm and Debian Trixie, i had the same behavior of dns_get_record of PHP, which makes me quite sure, that this is not a glibc oder PHP problem, and therefore most likely also not caused by the upgrade from Bookworm to Trixie.

And in Nextclouds DnsMiddleware, i can see no relevant code changes for over a year (just some linting fixes, nothing special).

Are you really sure, that you...

  1. Didnt added a search domain to your system
  2. Didnt changed the search domain of your system
  3. You or any other person with access to the DNS settings of the search console added a AAAA wildcard record

Please note, that the search domain of the system may change dynamically, if you use DHCP.

I am very sure, that it was pure "luck" and randomness, that the effect came up with Debian Trixie and Nextcloud 32 in your situation.

WhoAmI0501 avatar Dec 06 '25 21:12 WhoAmI0501

@WhoAmI0501:

Are you really sure, that you...

  1. Didnt added a search domain to your system
  2. Didnt changed the search domain of your system

When it comes to the whole system, yes, pretty sure:

~# ls -l /etc/resolv.conf
-rw-r--r-- 1 root root 68 27. Mai 2018  /etc/resolv.conf

Nextcloud runs in php-fpm, but I'm not using its chroot feature, so I cannot imagine how a different resolver configuration may come into play here - though that might be possible. Any hints / ideas appreciated.

Trixie changed php from 8.2 to 8.4, and brought a significant curl update (and thus probably libcurl) also mentioned in the Debian 13 release notes - though it does not indicate significant changes to name resolution: https://www.debian.org/releases/trixie/release-notes/whats-new.de.html#wcurl-and-http-3-support-in-curl

  1. You or any other person with access to the DNS settings of the search console added a AAAA wildcard record

What do you mean by "search console" - was it a typo and you meant "search domain"?

I'm the only one administrating the zone relevant for the search domain, and its current zone file is from July 31. So no changes in this regard either.

If you're referring to something else with "search console", I'm not aware of what it means and need some explanation. I'm always changing the DNS configuration in the bind zone files using a text editor.

gohrner avatar Dec 07 '25 12:12 gohrner

What do you mean by "search console" - was it a typo and you meant "search domain"?

It was a typo. I meant search domain.

and brought a significant curl update (and thus probably libcurl)

This is true, but Nextcloud is doing the DNS resolution itself via its Middleware and passes the IP addresses to libcurl, so libcurl isn't doing any DNS related stuff in this situation, at least from what i have seen.

I also compared the glibc code of getnameinfo, which is the method that is called by PHP if dns_get_record is executed, and i didnt see any major changes, which would explain the behavior change, between Debian Bookworm (2.36) and Trixie (2.41).

Have you tried my "sorting out patch" for the DnsMiddleware from my initial comment? You may also want to add some var_dump's to your DnsMiddleware, if you want to debug the behavior on your box yourself and compare the behavior of the current situation and if you comment out the search entry in /etc/resolv.conf. While doint that, take care of the caching behavior of your DnsMiddleware, i had to flush my Redis Cache multiple times manually, to see "real" DNS responses.

This would at least help us to find our, whether we suffer from the same problem or reason, or if we see entirely different error patterns.

WhoAmI0501 avatar Dec 08 '25 19:12 WhoAmI0501