KTOR-5199 Support WebSockets in Curl engine

[dtretyakov]

trafficstars

Subsystem Client, Curl engine

Motivation Curl 7.86.0 added experimental support for WebSockets.

Solution This PR brings support of experimental WebSockets in libcurl KTOR-5199.

To verify WebSockets availability we're using curl_version_info which returns list of enabled protocols.

The CurlResponseBodyData become interface with CurlHttpResponseBody/CurlWebSocketResponseBody implementations.

The bodyStartedReceiving is used to detect the end of headers section.

We're using WebSocket with callbacks approach where:

• curl_ws_send used to send outgoing frames
• onBodyChunkReceived with curl_ws_meta used to receive incoming frames

Environment Since WebSockets feature is experimental we need to enable them during curl compliation.

macOS

curl https://raw.githubusercontent.com/dtretyakov/homebrew-cloudflare/master/curl.rb -o curl.rb
brew install -s curl.rb 

linux

• Reference
• Commit
• Workflow file

[DRSchlaubi]

Trying to connect to wss://gateway.discord.gg/?v=10&encoding=json&compress=zlib-stream is returning 400 Bad request so there are some issues with this implementation`

After some further debugging the Sec-WebSocket-Key and Sec-WebSocket-Version headers are set twice, once by curl and once by ktor, removing them form ktor solves the 400 Bad request issue

New issue curl_multi_perform seems to block until the connection is done, making any additional requests impossible

[DRSchlaubi]

Possible fix for duplicated header issue

Subject: [PATCH] Fix duplicate headers
---
Index: ktor-client/ktor-client-curl/desktop/src/io/ktor/client/engine/curl/internal/CurlAdapters.kt
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/ktor-client/ktor-client-curl/desktop/src/io/ktor/client/engine/curl/internal/CurlAdapters.kt b/ktor-client/ktor-client-curl/desktop/src/io/ktor/client/engine/curl/internal/CurlAdapters.kt
--- a/ktor-client/ktor-client-curl/desktop/src/io/ktor/client/engine/curl/internal/CurlAdapters.kt	(revision 6265a80481d77cdbb2ff7950750e8ee5aa085813)
+++ b/ktor-client/ktor-client-curl/desktop/src/io/ktor/client/engine/curl/internal/CurlAdapters.kt	(date 1706196036063)
@@ -71,8 +71,11 @@
     var result: CPointer<curl_slist>? = null
 
     mergeHeaders(headers, body) { key, value ->
-        val header = "$key: $value"
-        result = curl_slist_append(result, header)
+        // These are set by curl itself
+        if (key != "Sec-WebSocket-Key" && key != "Sec-WebSocket-Version") {
+            val header = "$key: $value"
+            result = curl_slist_append(result, header)
+        }
     }
 
     result = curl_slist_append(result, "Expect:")

[dtretyakov]

@DRSchlaubi thanks a lot for verification and review.

New issue curl_multi_perform seems to block until the connection is done, making any additional requests impossible

Maybe you could share a test case when it could be reproduced?

Trying to connect to wss://gateway.discord.gg/?v=10&encoding=json&compress=zlib-stream Possible fix for duplicated header issue

Thanks, I added a list of WebSocket headers which are handled by cURL itself, so connection to this service should work.

[DRSchlaubi]

Maybe you could share a test case when it could be reproduced?

Sure

fun main() = runBlocking { 
    val client = HttpClient { 
        install(WebSockets)
    }
    
    launch { 
        client.webSocket("wss://echo.websocket.org") {
            while(!incoming.isClosedForReceive) {
              // Keep connection alive
                incoming.receive()
            }
        }
    }
    
    println("Wait for websocket to connect")
    delay(10.seconds)
    println("Requesting now!")
    
    println(client.get("https://httpbin.org/200"))
}

[DRSchlaubi]

~~After the latest changes most headers are not set, including "Authorization", which is not great~~

Fix in review comment

[DRSchlaubi]

After more debugging it seems like the issue is this loop

    @OptIn(ExperimentalForeignApi::class)
    internal fun perform() {
        if (activeHandles.isEmpty()) return

        memScoped {
            val transfersRunning = alloc<IntVar>()
            do {
                println("Transfers running:${transfersRunning.value}")
                synchronized(easyHandlesToUnpauseLock) {
                    var handle = easyHandlesToUnpause.removeFirstOrNull()
                    while (handle != null) {
                        curl_easy_pause(handle, CURLPAUSE_CONT)
                        handle = easyHandlesToUnpause.removeFirstOrNull()
                    }
                }
                println("Running curl_multi_perform")
                curl_multi_perform(multiHandle, transfersRunning.ptr).verify()
                println("Ran curl_multi_perform")
                if (transfersRunning.value != 0) {
                    curl_multi_poll(multiHandle, null, 0.toUInt(), 10000, null).verify()
                    println("Ran curl_multi_poll")
                }
                if (transfersRunning.value < activeHandles.size) {
                    handleCompleted()
                    println("Ran complete")
                }
                println("Loop done")
            } while (transfersRunning.value != 0)
        }
    }

Because while (transfersRunning.value != 0) will remain true as long as the ws connection is still active

[dtretyakov]

After more debugging it seems like the issue is this loop Because while (transfersRunning.value != 0) will remain true as long as the ws connection is still active

@DRSchlaubi hopefully it was also addressed in the last commit, so you could try updating.

[DRSchlaubi]

At first glance your new test case should cover this, will test myself tmr :+1:

[DRSchlaubi]

That indeed fixed it

[dtretyakov]

Please let me know when you have an update about the Curl binary version. I will run tests on CI

Currently we're waiting when static linking of libcurl with it's dependencies will be in the repository for all targets and after that could merge the changes.

• Prototype: https://github.com/dtretyakov/ktor/commit/3c4dcd678990f2c7054bc0d6c3c4c92b49202de5
• Build scripts which could be used to create static libraries for linux/macOS: https://github.com/stunnel/static-curl

[DRSchlaubi]

Any updates

[git-bruh]

Hey @dtretyakov , thanks a lot for your work on this PR. I noticed some strange behaviour when receiving payloads greater than 1024 bytes and it seems to be a libcurl quirk, have raised a PR here: https://github.com/dtretyakov/ktor/pull/1