NoRouteToHostException when making a request to https url from a zio-http Client using Scala 3

[lachezar]

I am getting weird error when making a request towards any https website when executing unit test (sbt test), however I don't get the same error when running my application (sbt run). I use zio 2.0.15, scala 3.3.0, zio-http 3.0.0-rc2, sbt 1.9.0 and Java 11.0.19.

Here is the code that you can use to reproduce the issue:

import zio.*
import zio.http.*
import zio.test.*

object ATest extends ZIOSpecDefault:

  val url = URL.decode("https://google.com/").toOption.get

  def spec = suite("ATest")(
    test("test") {
      ZIO
        .serviceWithZIO[Client](_.request(Request.get(url)))
        .map(res => assert(res)(Assertion.anything))
    }
  ).provideShared(Client.default)

The error I get is

Exception in thread "zio-fiber-113,110" io.netty.channel.AbstractChannel$AnnotatedNoRouteToHostException: null: google.com/2a00:1450:400f:802:0:0:0:200e:443
    	at ATest.spec(ATest.scala:14)
    	at ATest.spec(ATest.scala:15)
    	at ATest.spec(ATest.scala:16)
    	Suppressed: java.net.NoRouteToHostException: null
    		at io.netty.channel.unix.Errors.newConnectException0(Errors.java:158)
    		at io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131)
    		at io.netty.channel.unix.Socket.connect(Socket.java:351)
    		at io.netty.channel.kqueue.AbstractKQueueChannel.doConnect0(AbstractKQueueChannel.java:727)
    		at io.netty.channel.kqueue.KQueueSocketChannel.doConnect0(KQueueSocketChannel.java:102)
    		at io.netty.channel.kqueue.AbstractKQueueChannel.doConnect(AbstractKQueueChannel.java:712)
    		at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.connect(AbstractKQueueChannel.java:558)
    		at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1342)
    		at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:653)

I have noticed that there is a unit test called ClientHttpsSpec that has been set as ignored for the past 2 years and I believe that it might shed more light of the issue https://github.com/zio/zio-http/pull/716

[sky0hunter]

I've actually the 'flaky' situation, my test calls 3 different https endpoints and if I run the test multiple times it can either be succesfull, or any of the 3 calls can fail with NoRouteToHost.

[jdegoes]

/bounty $100

[algora-pbc[bot]]

💎 $100 bounty • ZIO

Steps to solve:

1. Start working: Comment /attempt #2280 with your implementation plan
2. Submit work: Create a pull request including /claim #2280 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to zio/zio-http!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @lackhoa	Sep 29, 2023, 1:06:27 PM	WIP

[sky0hunter]

The cause of this issue was for me, that I had no IPv6 support on my router. ( I.e. setting -Djava.net.preferIPv4Stack=true fixed the issue ). It seems that the DnsNameResolver can return a IPv6 address if the preferIPv4Stack is not set. ( https://github.com/netty/netty/issues/5657 )

[lachezar]

@sky0hunter it seems that this flag really solves the issue for me at least 🎆

[dkovalenko]

Should we default to IPv4 in this case? I had similar problems with this even with the "http".

io.netty.channel.AbstractChannel$AnnotatedNoRouteToHostException: null: google.com/2a00:1450:401b:80e:0:0:0:200e:80

We could make a IPv4 as default and then have an option to enable IPv6. @jdegoes WDYT?

[carlos-verdes]

I tried with the flag mentioned but still I see HTTP client trying to connect using IPv6

Any way to setup with ClientConfig directly inside ZIO?

I'm talking on something like this part:

  val sslConfig = ClientSSLConfig.Default
  val clientConfig = ZClient.Config.default.ssl(sslConfig)

... but including an option like config.enforceIpv4(true)

[lackhoa]

/attempt #2280 Lol I'm trying to optimize another issue but this stands in the way... So I'll do it too! I encounter this issue using Scala 2, and my Macbook is perfectly capable of using ipv6 (update: no it can't, my ISP doesn't support ipv6 it seems :)).

Options

• Cancel my attempt

[carlos-verdes]

I don't know why the flag is not working on SBT but I added this into my main scala file and I at least the issue is not happening anymore.

java.lang.System.setProperty("java.net.preferIPv4Stack", "true")

[alvinj]

If it helps to know it, I was intermittently seeing the same error message when using:

• ZIO 2.0.22
• ZIO HTTP 3.0.0-RC4
• Scala 3.3.3
• macOS 14.4.1 system
• SBT 1.5.5

and the property settings shown in this issue resolved the problem.

[Nojipiz]

More information about this error:

• ZIO 2.0.22
• ZIO HTTP 3.0.0-RC6
• Scala 3.3.3
• SBT 1.9.9

The property settings:

java.lang.System.setProperty("java.net.preferIPv4Stack", "true")

Just makes it fail with another error:

io.netty.channel.AbstractChannel$AnnotatedConnectException: connect(..) failed: Address family not supported by protocol: google.com/[2a01:ad00:1:3:0:0:0:183]:443)

[FabioPinheiro]

But why this seems to be only a Problem when running the test? May it because I run normally run inside of Docker image it doesn't support IPv6 but when developing locally the tests runs outside on a JVM in the host machine.

For now I'll only config the module in build.sbt with

  .jvmSettings(
    Test / fork := true,
    Test / javaOptions := Seq("-Djava.net.preferIPv4Stack=true"),
  )