sttp icon indicating copy to clipboard operation
sttp copied to clipboard

Fix native on ARM (Macs)

Open szymon-rd opened this issue 1 year ago • 5 comments

Fix #1668

Something was going wrong with the way libcurl exposes API functions with varargs, and how scala native was calling them on ARM Macs. So I added an interface in between. (thanks @WojciechMazur)

szymon-rd avatar Feb 22 '24 16:02 szymon-rd

If you rebase, the JS build problems should get fixed

adamw avatar Feb 26 '24 15:02 adamw

bumping for subscription

lbialy avatar Mar 25 '24 15:03 lbialy

@szymon-rd I switched scala-native to 0.5.1 and applied a few more updates, could you verify if this PR fixes the ARM issue now?

kciesielski avatar Apr 24 '24 14:04 kciesielski

Seems to be building fine on ARM (MacOS M1), but I'm experiencing a deadlocks in some tests, eg. in the redirects. What's strange is the fact that I can observe it not only in this PR, but also on GitPod env (x64) on both this PR and main branch. I'm not really sure why. After attaching debugger the libcurl seems to be stuck in fread functions, although it seems like it should be done already (it was able to process the headers, there should be no body?)

WojciechMazur avatar Apr 27 '24 23:04 WojciechMazur

I don't have access to ARM Mac now so sadly I am not able to verify it. @kciesielski

szymon-rd avatar May 06 '24 09:05 szymon-rd

I've figured out the issues with the deadlocks - the empty POST request in some (newer?) versions of curl by default might wait for input on stdin. To prevent that we should set PostFields options with empty input explicitly. After applying this change I was able to successfully run all tests on MacOS M1. I've tried to setup a CI job for sttp to use macos-14 runner - these are using ARM64 arch on M1 chips, however there are some issues when using native libraries. For some reason it expects x86_64 libs, instead of ARM64, it's really strange.

Anyway I here's a fix and a small cleanup. I've cannot push it directly to the PR. @kciesielski can you commit this patch to speed up merging this PR?

diff --git a/core/src/main/scalanative/sttp/client4/curl/AbstractCurlBackend.scala b/core/src/main/scalanative/sttp/client4/curl/AbstractCurlBackend.scala
index 337f9a266..d7e657bc7 100644
--- a/core/src/main/scalanative/sttp/client4/curl/AbstractCurlBackend.scala
+++ b/core/src/main/scalanative/sttp/client4/curl/AbstractCurlBackend.scala
@@ -117,7 +117,7 @@ abstract class AbstractCurlBackend[F[_]](_monad: MonadError[F], verbose: Boolean
     curl.option(HeaderData, spaces.headersResp)
     curl.option(Url, request.uri.toString)
     setMethod(curl, request.method)
-    setRequestBody(curl, request.body)
+    setRequestBody(curl, request.body, request.method)
     monad.flatMap(perform(curl)) { _ =>
       curl.info(ResponseCode, spaces.httpCode)
       val responseBody = fromCString((!spaces.bodyResp)._1)
@@ -157,7 +157,7 @@ abstract class AbstractCurlBackend[F[_]](_monad: MonadError[F], verbose: Boolean
     curl.option(WriteData, outputFilePtr)
     curl.option(Url, request.uri.toString)
     setMethod(curl, request.method)
-    setRequestBody(curl, request.body)
+    setRequestBody(curl, request.body, request.method)
     monad.flatMap(perform(curl)) { _ =>
       curl.info(ResponseCode, spaces.httpCode)
       val httpCode = StatusCode((!spaces.httpCode).toInt)
@@ -195,7 +195,7 @@ abstract class AbstractCurlBackend[F[_]](_monad: MonadError[F], verbose: Boolean
     lift(m)
   }
 
-  private def setRequestBody(curl: CurlHandle, body: GenericRequestBody[R])(implicit
+  private def setRequestBody(curl: CurlHandle, body: GenericRequestBody[R], method: Method)(implicit
       ctx: Context
   ): F[CurlCode] = {
     implicit val z = ctx.zone
@@ -223,7 +223,9 @@ abstract class AbstractCurlBackend[F[_]](_monad: MonadError[F], verbose: Boolean
       case StreamBody(_) =>
         monad.error(new IllegalStateException("CurlBackend does not support stream request body"))
       case NoBody =>
-        monad.unit(CurlCode.Ok)
+        // POST with empty body might wait for the input on stdin
+        if (method.is(Method.POST)) lift(curl.option(PostFields, c""))
+        else monad.unit(CurlCode.Ok)
     }
   }
 
diff --git a/core/src/main/scalanative/sttp/client4/curl/internal/CurlApi.scala b/core/src/main/scalanative/sttp/client4/curl/internal/CurlApi.scala
index 16f592e6c..5b5a17e92 100644
--- a/core/src/main/scalanative/sttp/client4/curl/internal/CurlApi.scala
+++ b/core/src/main/scalanative/sttp/client4/curl/internal/CurlApi.scala
@@ -4,7 +4,6 @@ import sttp.client4.curl.internal.CurlCode.CurlCode
 import sttp.client4.curl.internal.CurlInfo.CurlInfo
 import sttp.client4.curl.internal.CurlOption.CurlOption
 
-import scala.scalanative.runtime.Boxes
 import scala.scalanative.unsafe.{Ptr, _}
 import scala.scalanative.unsigned._
 import java.nio.charset.StandardCharsets
@@ -29,22 +28,22 @@ private[client4] object CurlApi {
     def cleanup(): Unit = CCurl.cleanup(handle)
 
     def option(option: CurlOption, parameter: String)(implicit z: Zone): CurlCode =
-      CurlCode(CCurl.setoptPtr(handle, option.id, toCString(parameter, StandardCharsets.UTF_8)))
+      this.option(option, toCString(parameter, StandardCharsets.UTF_8))
 
-    def option(option: CurlOption, parameter: Long)(implicit z: Zone): CurlCode =
+    def option(option: CurlOption, parameter: Long): CurlCode =
       CurlCode(CCurl.setoptLong(handle, option.id, parameter))
 
-    def option(option: CurlOption, parameter: Int)(implicit z: Zone): CurlCode =
+    def option(option: CurlOption, parameter: Int): CurlCode =
       CurlCode(CCurl.setoptInt(handle, option.id, parameter))
 
-    def option(option: CurlOption, parameter: Boolean)(implicit z: Zone): CurlCode =
-      CurlCode(CCurl.setoptInt(handle, option.id, if (parameter) 1 else 0))
+    def option(option: CurlOption, parameter: Boolean): CurlCode =
+      this.option(option, if (parameter) 1 else 0)
 
     def option(option: CurlOption, parameter: Ptr[_]): CurlCode =
       CurlCode(CCurl.setoptPtr(handle, option.id, parameter))
 
-    def option[FuncPtr <: CFuncPtr](option: CurlOption, parameter: FuncPtr)(implicit z: Zone): CurlCode =
-      CurlCode(CCurl.setoptPtr(handle, option.id, Boxes.boxToPtr[Byte](Boxes.unboxToCFuncPtr0(parameter))))
+    def option(option: CurlOption, parameter: CFuncPtr): CurlCode =
+      this.option(option, CFuncPtr.toPtr(parameter))
 
     def info(curlInfo: CurlInfo, parameter: Long)(implicit z: Zone): CurlCode = {
       val lPtr = alloc[Long](sizeof[Long])

WojciechMazur avatar May 09 '24 19:05 WojciechMazur

Sure, I'm on it. Thanks a lot for this investigation @WojciechMazur!

kciesielski avatar May 10 '24 11:05 kciesielski