swift-android-sdk icon indicating copy to clipboard operation
swift-android-sdk copied to clipboard

Support 16KB pages size

Open marcprux opened this issue 6 months ago • 30 comments

Just like @andriydruk did at https://github.com/readdle/swift-android-toolchain/commit/dc50b08f5b67c921d7b710c3709ed64e6b15df76

marcprux avatar May 16 '25 22:05 marcprux

I don't intend to ever support this new feature with my 6.1 releases, and hopefully we can switch to official releases after that, so best to file this upstream.

finagolfin avatar May 17 '25 07:05 finagolfin

I filed a PR for this at https://github.com/swiftlang/swift/pull/81596

marcprux avatar May 18 '25 23:05 marcprux

Depending on whether #221 works, I'll add this here.

finagolfin avatar Jun 21 '25 09:06 finagolfin

Most libraries should now work with 1b95949, will close once the last few are modified.

finagolfin avatar Jul 21 '25 05:07 finagolfin

With the recent 6.2 release, all bundles here are 16KB page-aligned.

finagolfin avatar Sep 28 '25 09:09 finagolfin

I've switched to the 6.2 release and am encountering an issue when launching the app on an Android Emulator with 16KB page size:

FATAL EXCEPTION: main
    Process: com.some.app, PID: 2779
    java.lang.UnsatisfiedLinkError: dlopen failed: "/data/app/~~0K2WnG4RLu3-5KL59-e7og==/com.some.app-k_D47KFQFRSMZ3KVQJK2kg==/base.apk!/lib/arm64-v8a/libandroid-spawn.so" failed to setup 16KiB App Compat
    at java.lang.Runtime.loadLibrary0(Runtime.java:1090)
    at java.lang.Runtime.loadLibrary0(Runtime.java:1012)
    at java.lang.System.loadLibrary(System.java:1765)
    at java.lang.Class.newInstance(Native Method)
    at android.app.AppComponentFactory.instantiateApplication(AppComponentFactory.java:76)
    at androidx.core.app.CoreComponentFactory.instantiateApplication(CoreComponentFactory.java:51)
    at android.app.Instrumentation.newApplication(Instrumentation.java:1347)
    at android.app.LoadedApk.makeApplicationInner(LoadedApk.java:1479)
    at android.app.LoadedApk.makeApplicationInner(LoadedApk.java:1411)
    at android.app.ActivityThread.handleBindApplication(ActivityThread.java:7790)
    at android.app.ActivityThread.-$$Nest$mhandleBindApplication(Unknown Source:0)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2546)
    at android.os.Handler.dispatchMessage(Handler.java:110)
    at android.os.Looper.loopOnce(Looper.java:248)
    at android.os.Looper.loop(Looper.java:338)
    at android.app.ActivityThread.main(ActivityThread.java:9067)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:593)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:932)

The app works fine on non-16KB emulators and physical devices. Is there a specific swift build flag needed to ensure the produced target's .so file is properly aligned for 16KB requirements?

MihaelIsaev avatar Sep 28 '25 10:09 MihaelIsaev

No, the bundle libraries have to be built on the CI that way, no subsequent build flag when using the bundle can fix that. I just checked and you are right that libandroid-spawn.so alone from this 6.2 bundle is now unaligned, which I had missed, but I get that from the Termux package repository and don't build it myself.

Is there any reason you're still using this SDK for deployment and not the new SDK that we've been posting about on the Swift forum? That will soon be the official SDK upstream, and I plan to stop releasing SDKs here after that. I will look at fixing my 6.2 SDK bundle here in the coming weeks, but you're better off using those linked SDKs.

finagolfin avatar Sep 28 '25 11:09 finagolfin

Is there any reason you're still using this SDK

Well, I just delayed switching to the new SDK while yours was working fine.

The new SDK is missing the following files that are present in your bundle:

libandroid-execinfo.so
libandroid-spawn.so
libc++_shared.so
libcharset.so
libcrypto.so
libcurl.so
libiconv.so
liblzma.so
libnghttp2.so
libnghttp3.so
libssh2.so
libssl.so
libxml2.so
libz.so

Is it because the new SDK assumes these files aren't needed for Swift apps?

My app is trying to load libandroid-spawn.so for some reason... guess maybe it is related to linking in this SDK

MihaelIsaev avatar Sep 28 '25 13:09 MihaelIsaev

Is it because the new SDK assumes these files aren't needed for Swift apps?

No, most of them aren't needed separately anymore with the upcoming official SDK because they're included elsewhere. libandroid-spawn is a backport of Bionic's posix_spawn wrapper to API 24, which is no longer needed because the official SDK will only support API 28 or later, when posix_spawn was added to Bionic. The rest you list should now be statically linked into libFoundation{Networking,XML}.so, try them out and let us know if we missed anything, as you just did with my bundle here. 😉

Since the upcoming official SDK builds everything from source itself, unlike my bundle here, we can always update it pretty quickly if anything goes wrong.

finagolfin avatar Sep 28 '25 17:09 finagolfin

Got it, thanks!

Trying to compile my app for the first time with the new official SDK and it fails with both https://github.com/PADL/AndroidLogging and https://github.com/PADL/AndroidLooper unfortunately:

Building for debugging...
[0/26] Write sources
In file included from 
.build/checkouts/AndroidLogging/Sources/CAndroidLogging/CAndroidLogging.cpp:1:
.build/checkouts/AndroidLogging/Sources/CAndroidLogging/include/CAndroidLogging.h:17:10: fatal error: 'android/log.h' file not found
   17 | #include <android/log.h>
      |          ^~~~~~~~~~~~~~~
1 error generated.
In file included from
.build/checkouts/AndroidLooper/Sources/CAndroidLooper/CAndroidLooper.cpp:17:
.build/checkouts/AndroidLooper/Sources/CAndroidLooper/include/CAndroidLooper.h:19:10: fatal error: 'android/looper.h' file not found
   19 | #include <android/looper.h>
      |          ^~~~~~~~~~~~~~~~~~
1 error generated.
[10/26] Compiling CAndroidLooper CAndroidLooper.cpp
[10/26] Compiling CAndroidLogging CAndroidLogging.cpp
In file included from
.build/checkouts/swift-system/Sources/CSystem/shims.c:12:
.build/checkouts/swift-system/Sources/CSystem/include/CSystemLinux.h:12:10: fatal error: 'sys/epoll.h' file not found
   12 | #include <sys/epoll.h>
      |          ^~~~~~~~~~~~~
[10/26] Compiling CSystem shims.c

Looks very very sad to me, don't know what to say. Both libs compiles fine with your SDK. This could be the answer why not to switch to the official one. But I have to since 16KB page size doesn't work in your SDK. In the new SDK it seems now I have to execute setup-android-sdk.sh... can't say that it is amazing UX. Your SDK is perfect because it just works out of the box, as it should, no extra steps.

MihaelIsaev avatar Sep 29 '25 00:09 MihaelIsaev

Trying to compile my app for the first time with the new official SDK and it fails

Did you follow the instructions there exactly, including running the setup-android-sdk.sh script to link the NDK to the upcoming official SDK? It looks like you did not, as it cannot find NDK headers.

If you did, this may be a bug in the upcoming official SDK, we'll look into it.

In the new SDK it seems now I have to execute setup-android-sdk.sh... can't say that it is amazing UX. Your SDK is perfect because it just works out of the box, as it should, no extra steps.

I ship the Android NDK sysroot with my SDK bundle, so you don't have to download the NDK yourself, which we didn't want to do with the upcoming official SDK. It's only two more steps the first time you install an official Android SDK, ie download the NDK then run a command to link the two, don't think it's a big deal. 😺

finagolfin avatar Sep 29 '25 04:09 finagolfin

Btw, I just tried cross-compiling those two PADL repos you linked with the upcoming official Android SDK, no problem. I'm guessing you either forgot to run the setup script or it gave you some error that something went wrong. I suggest you sdk remove it and start over.

finagolfin avatar Sep 29 '25 07:09 finagolfin

Btw, I just tried cross-compiling those two PADL repos you linked with the upcoming official Android SDK, no problem. I'm guessing you either forgot to run the setup script or it gave you some error that something went wrong. I suggest you sdk remove it and start over.

I'm glad someone is using these! :)

lhoward avatar Sep 29 '25 07:09 lhoward

wasn't aware of the new official one; testing it out now.

on the old one this would compile:

import Foundation

#if canImport(FoundationNetworking)
import FoundationNetworking
#endif

func testSocket() {
    var socketFd: Int32 = 0
    
    #if os(Android)
    socketFd = socket(AF_INET, SOCK_DGRAM, 0)
    #elseif os(Linux)
    socketFd = socket(AF_INET, Int32(SOCK_DGRAM.rawValue), 0)
    #else
    socketFd = socket(AF_INET, SOCK_DGRAM, 0)
    #endif
}
/Users/rjbowli/Desktop/test_socket/Sources/test_socket/test_socket.swift:13:16: error: cannot find 'socket' in scope
11 |     
12 |     #if os(Android)
13 |     socketFd = socket(AF_INET, SOCK_DGRAM, 0)
   |                `- error: cannot find 'socket' in scope
14 |     #elseif os(Linux)
15 |     socketFd = socket(AF_INET, Int32(SOCK_DGRAM.rawValue), 0)

I assuming this is more because of the Foundation rework than the android SDK, however I am having a hard time figuring out what exactly to import in order to get access to socket on Android now? On macos and linux I can do Darwin.socket and Glibc.socket. I tried Bionic.socket and got:

/Users/rjbowli/Desktop/test_socket/Sources/test_socket/test_socket.swift:7:20: error: module 'Bionic' has no member named 'socket'
 5 | #endif
 6 | 
 7 | let posix_socket = Bionic.socket
   |                    `- error: module 'Bionic' has no member named 'socket

KittyMac avatar Sep 29 '25 15:09 KittyMac

I think you want to add an import header like:

#if canImport(Darwin)
import Darwin
#elseif canImport(Android)
import Android
#elseif canImport(Glibc)
import Glibc
#else
#error("Unknown platform")
#endif

See https://skip.tools/blog/android-native-swift-packages/#importing-the-android-module for some more details and advice.

Also, here's an example of some Android-aware code that handles socket() on different platforms: https://github.com/swiftlang/swift-corelibs-foundation/blob/94ef6ab1e680bd342897671106ed3ba79ba89d70/Tests/Foundation/HTTPServer.swift#L108-L122

marcprux avatar Sep 29 '25 15:09 marcprux

great, my swift project roughly compile now. I currently use the android sdk to built a dynamic library to include in my android app. I notice above the comment about static linkage, so I tried both.

when I compile for dynamic library, I get a dynamic library with dynamic dependencies, which would be the expected result:

Dynamic section at offset 0x2d87a8 contains 43 entries:
  Tag        Type                         Name/Value
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN]
 0x0000000000000001 (NEEDED)             Shared library: [libswiftCore.so]
 0x0000000000000001 (NEEDED)             Shared library: [libswift_Concurrency.so]
 0x0000000000000001 (NEEDED)             Shared library: [libswift_StringProcessing.so]

when I compile it static I get a static library, but when I try and compile that into my Android project I get a bunch of errors like:

undefined symbol: swift_retain
undefined symbol: swift_initStackObject
undefined symbol: swift_release
undefined symbol: swift_bridgeObjectRetain
....

if I continue with the dynamic route, where do the new SDK's shared .so files live?

File size for the my native libraries is a concern for us, so I assume static linkage might get those smaller; any idea what am I missing to getting the static lib working?


Ok, I misread the above and that only the dependencies for libFoundationNetworking are statically linked in to it.

I tried building with the following and the size of my .a doesn't change, not sure how to generate a .a from my swift project which has everything compiled in (if that's even possible)

swiftly run swift build  --configuration=release --static-swift-stdlib --swift-sdk aarch64-unknown-linux-android28 +6.2

On the dynamic front I found the .so files in the android 6.2 sdk artifactbundle; it looks like the sizes of these are significantly larger than they used to be. My back-of-napkin-math says that if I switch over to this the size of my native libraries will easily be 2-3x larger than what I am using now. For reference I am currently using a 5.8 version of the android sdk with 16kb page support already added.

KittyMac avatar Sep 29 '25 16:09 KittyMac

I tried building with the following and the size of my .a doesn't change, not sure how to generate a .a from my swift project which has everything compiled in

That option is not provided by SwiftPM. What it does is optionally link the static runtime libraries in for executables, but doing the same with library products has not been implemented.

On the dynamic front I found the .so files in the android 6.2 sdk artifactbundle; it looks like the sizes of these are significantly larger than they used to be. My back-of-napkin-math says that if I switch over to this the size of my native libraries will easily be 2-3x larger than what I am using now.

I doubt that, as we are still mostly compiling the same code, except adding the Foundation rewrite and the Testing library, which isn't needed for deployment. In fact, if you don't need internationalization, you can ship much less from Foundation now and it should be significantly smaller to deploy.

For reference I am currently using a 5.8 version of the android sdk

5.8?! Why so old?

finagolfin avatar Oct 07 '25 11:10 finagolfin

I doubt that, as we are still mostly compiling the same code, except adding the Foundation rewrite and the Testing library, which isn't needed for deployment. In fact, if you don't need internationalization, you can ship much less from Foundation now and it should be significantly smaller to deploy.

Great! I did a deeper dive, created minimal builds with both my 5.8 and new 6.2. After stripping and compiling with optimizations etc I see the new stuff is only about 22% larger than my current build pipeline.

I imagine new lib_FoundationICU.so is statically linking a full libicudata to make it 38.7MB? I don't need internationalization where/how is the best approach to getting the stripped versions of these?

drwxr-xr-x  27 user  staff       864 Oct  7 11:31 .
drwxr-xr-x   5 user  staff       160 Oct  7 11:31 ..
-rw-------   1 user  staff    134344 Oct  7 11:31 libandroid-spawn.so
-rw-r--r--   1 user  staff      8400 Oct  7 11:31 libBlocksRuntime.so
-rwxr-xr-x   1 user  staff   1492200 Oct  7 11:31 libc++_sharedSR.so
-rwx------   1 user  staff   5173664 Oct  7 11:31 libcryptoSR.so
-rwx------   1 user  staff   1061936 Oct  7 11:31 libcurl.so
-rw-r--r--   1 user  staff    465824 Oct  7 11:31 libdispatch.so
-rw-r--r--   1 user  staff   9224736 Oct  7 11:31 libFoundation.so
-rw-r--r--   1 user  staff   1382336 Oct  7 11:31 libFoundationNetworking.so
-rwxr-xr-x   1 user  staff    306664 Oct  7 11:31 libHitch.so
-rwx------   1 user  staff  13173520 Oct  7 11:31 libicudata.so
-rwx------   1 user  staff   5561344 Oct  7 11:31 libicui18n.so
-rwx------   1 user  staff   2740736 Oct  7 11:31 libicuuc.so
-rwx------   1 user  staff    159056 Oct  7 11:31 libnghttp2.so
-rwx------   1 user  staff    148112 Oct  7 11:31 libnghttp3.so
-rwxr-xr-x   1 user  staff     67248 Oct  7 11:31 libSilkRoadFramework.so
-rwx------   1 user  staff    400768 Oct  7 11:31 libssh2.so
-rwx------   1 user  staff   1037456 Oct  7 11:31 libsslSR.so
-rwxr-xr-x   1 user  staff    811672 Oct  7 11:31 libswift_Concurrency.so
-rwxr-xr-x   1 user  staff   2005520 Oct  7 11:31 libswift_RegexParser.so
-rwxr-xr-x   1 user  staff   1020576 Oct  7 11:31 libswift_StringProcessing.so
-rwxr-xr-x   1 user  staff   7737552 Oct  7 11:31 libswiftCore.so
-rw-r--r--   1 user  staff    320872 Oct  7 11:31 libswiftDispatch.so
-rwxr-xr-x   1 user  staff    154376 Oct  7 11:31 libswiftGlibc.so
-rwx------   1 user  staff   1114736 Oct  7 11:31 libxml2.so
-rwx------   1 user  staff    202336 Oct  7 11:31 libzSR.so

... 55.9 MB


drwxr-xr-x  27 user  staff       864 Oct  7 12:20 .
drwxr-xr-x   6 user  staff       192 Oct  7 11:34 ..
-rw-r--r--@  1 user  staff      6148 Oct  7 12:10 .DS_Store
-rw-r--r--   1 user  staff  38693520 Oct  7 12:20 lib_FoundationICU.so
-rw-r--r--   1 user  staff     10888 Oct  7 12:20 libBlocksRuntime.so
-rwxr-xr-x   1 user  staff   1292440 Oct  7 12:20 libc++_shared.so
-rw-r--r--   1 user  staff    371672 Oct  7 12:20 libdispatch.so
-rw-r--r--   1 user  staff   6198248 Oct  7 12:20 libFoundation.so
-rw-r--r--   1 user  staff   6139576 Oct  7 12:20 libFoundationEssentials.so
-rw-r--r--   1 user  staff   1713512 Oct  7 12:20 libFoundationInternationalization.so
-rw-r--r--   1 user  staff   3478680 Oct  7 12:20 libFoundationNetworking.so
-rwxr-xr-x   1 user  staff    247064 Oct  7 12:20 libHitch.so
-rwxr-xr-x   1 user  staff      8832 Oct  7 12:20 libSilkRoadFramework.so
-rwxr-xr-x   1 user  staff      7248 Oct  7 12:20 libswift_Builtin_float.so
-rwxr-xr-x   1 user  staff    890344 Oct  7 12:20 libswift_Concurrency.so
-rwxr-xr-x   1 user  staff    373016 Oct  7 12:20 libswift_Differentiation.so
-rwxr-xr-x   1 user  staff      6616 Oct  7 12:20 libswift_math.so
-rwxr-xr-x   1 user  staff    978256 Oct  7 12:20 libswift_RegexParser.so
-rwxr-xr-x   1 user  staff    683296 Oct  7 12:20 libswift_StringProcessing.so
-rwxr-xr-x   1 user  staff      8184 Oct  7 12:20 libswift_Volatile.so
-rwxr-xr-x   1 user  staff     47864 Oct  7 12:20 libswiftAndroid.so
-rwxr-xr-x   1 user  staff   6806864 Oct  7 12:20 libswiftCore.so
-rw-r--r--   1 user  staff    171744 Oct  7 12:20 libswiftDispatch.so
-rwxr-xr-x   1 user  staff    105984 Oct  7 12:20 libswiftDistributed.so
-rwxr-xr-x   1 user  staff     75872 Oct  7 12:20 libswiftObservation.so
-rwxr-xr-x   1 user  staff     82840 Oct  7 12:20 libswiftRegexBuilder.so
-rwxr-xr-x   1 user  staff     61640 Oct  7 12:20 libswiftSynchronization.so

... 68.5 MB

5.8?! Why so old?

Nothing too obtuse; 5.8 currently works great and nothing new in Swift really worth updating for.

KittyMac avatar Oct 07 '25 16:10 KittyMac

The size of FoundationInternationalization is indeed problematic. We (the Android WG) have discussed how we might address it, and one option is to externalize the data files from the shared object itself (see https://forums.swift.org/t/android-app-size-and-lib-foundationicu-so/78399 and https://github.com/swiftlang/swift-foundation-icu/pull/53), but there hasn't been much work done on it recently.

I don't need internationalization where/how is the best approach to getting the stripped versions of these?

Can you get by with just FoundationEssentials? If so, your libraries won't need libFoundationInternationalization.so. E.g.:

#if canImport(FoundationEssentials)
import FoundationEssentials // e.g., Android
#else
import Foundation // e.g., Darwin
#endif

However, this precludes using FoundationNetworking (which relies on FoundationInternationalization), so excluding it is a non-starter for all but the simplest uses of Swift on Android in my experience.

marcprux avatar Oct 07 '25 18:10 marcprux

one option is to externalize the data files from the shared object itself

This is kind of what we had before when it was dynamically linked; I was able to build english-only using the termux build scripts and simply replace it.

Other options to consider could be:

  1. build two lib_FoundationICU.so to include in the SDK, the default one as is and another, slimmer one. Since we need to copy it out anyway to use it the ppl who want the slimmer can just copy that.
  2. provide build instructions and/or ease the build process so that we can provide our own filterfile or whatever options we need to slim it

1 would be more convenient but the choices to slim not work for everyone 2 is fine enough because this would only need to be done when changing to new versions of the toolchain

Can you get by with just FoundationEssentials?

I didn't know about that but I do use FoundationNetworking.

KittyMac avatar Oct 07 '25 18:10 KittyMac

Other options to consider could be:

FoundationICU is now a Swift package, so it is much easier to hack on than the upstream build used before: go for it. 😸

finagolfin avatar Oct 07 '25 19:10 finagolfin

FoundationICU is now a Swift package, so it is much easier to hack on than the upstream build used before: go for it. 😸

oh... OOOOHHHH! i just build it with the android sdk and replace; nice!

KittyMac avatar Oct 07 '25 19:10 KittyMac

That worked swell; as a quick test I took the "minimal" data files from https://github.com/GoodNotes/swift-icudata-slim/tree/0.2.0?tab=readme-ov-file and replaced the current embedded data in lib_FoundationICU.so; file size dropped to 5.7MB.

KittyMac avatar Oct 07 '25 21:10 KittyMac

Wow, I hadn't heard about that project! I didn't think that we would be able to use the standard ICU data files and the ICU Data Build Tool, since Foundation relies on some Apple-specific derivation of the format. I'm glad to hear it works.

What happens when you try to perform an i18n operation on a locale whose information has been trimmed? Does it just fall back to English, or give an error (or return nil or crash)?

I still think the best way forward is to load the data files at runtime so we can offer the developer the option on how to trim their data files when they package their app, but it is heartening to know that there is at least a path forward for a sufficiently-motivated developer to slim down their packages for distribution.

marcprux avatar Oct 07 '25 21:10 marcprux

didn't have much time so I had AI generate some test code and I simply ran it against both the original in the android sdk and the "minimal" from the other project; no crashes just nils in some places:

// original lib_FoundationICU.so
21:10:39.770  E  TAG: Locale fr_FR: currencyCode = Optional("EUR"), symbol = Optional("€"), language = Optional("fr"), region = Optional("FR")
21:10:39.772  E  TAG: Locale zh_Hant_TW: currencyCode = Optional("TWD"), symbol = Optional("$"), language = Optional("zh"), region = Optional("TW")
21:10:39.772  E  TAG: Locale de_DE: currencyCode = Optional("EUR"), symbol = Optional("€"), language = Optional("de"), region = Optional("DE")
21:10:39.774  E  TAG: Locale en_US: currencyCode = Optional("USD"), symbol = Optional("$"), language = Optional("en"), region = Optional("US")
21:10:39.778  E  TAG: TimeZone America/New_York: secondsFromGMT = -14400, abbreviation = Optional("GMT-4")
21:10:39.778  E  TAG: TimeZone Europe/Paris: secondsFromGMT = 7200, abbreviation = Optional("GMT+2")
21:10:39.779  E  TAG: TimeZone Asia/Tokyo: secondsFromGMT = 32400, abbreviation = Optional("GMT+9")
21:10:39.779  E  TAG: TimeZone UTC: secondsFromGMT = 0, abbreviation = Optional("GMT")
21:10:39.780  E  TAG: NY is DST? true
21:10:39.780  E  TAG: s.data(using:) returned nil for encoding Western (ISO Latin 1)
21:10:39.782  E  TAG: Hello — café Hello — café Round-trip encoding failed for encoding Western (Windows Latin 1)
21:10:39.782  E  TAG: s.data(using:) returned nil for encoding Western (ASCII)
21:10:39.802  E  TAG: After toLatin: éÅß中文, did: nil
21:10:39.805  E  TAG: localizedStandardCompare: 1
21:10:39.805  E  TAG: NSString compare: 1
21:10:39.807  E  TAG: true By default Swift strings treat them as canonical equiv
21:10:39.807  E  TAG: Decomposed: é, Precomposed: é
21:10:39.808  E  TAG: é é Decomposition gave something unexpected
21:10:39.808  E  TAG: é é Precomposition gave something unexpected

// "minimal" 
21:15:36.732  E  TAG: Locale fr_FR: currencyCode = Optional("EUR"), symbol = Optional("€"), language = Optional("fr"), region = Optional("FR")
21:15:36.733  E  TAG: Locale zh_Hant_TW: currencyCode = nil, symbol = nil, language = Optional("zh"), region = Optional("TW")
21:15:36.736  E  TAG: Locale de_DE: currencyCode = Optional("EUR"), symbol = Optional("€"), language = Optional("de"), region = Optional("DE")
21:15:36.738  E  TAG: Locale en_US: currencyCode = nil, symbol = nil, language = Optional("en"), region = Optional("US")
21:15:36.741  E  TAG: TimeZone America/New_York: secondsFromGMT = -14400, abbreviation = nil
21:15:36.742  E  TAG: TimeZone Europe/Paris: secondsFromGMT = 7200, abbreviation = nil
21:15:36.742  E  TAG: TimeZone Asia/Tokyo: secondsFromGMT = 32400, abbreviation = nil
21:15:36.742  E  TAG: TimeZone UTC: secondsFromGMT = 0, abbreviation = nil
21:15:36.742  E  TAG: NY is DST? true
21:15:36.743  E  TAG: s.data(using:) returned nil for encoding Western (ISO Latin 1)
21:15:36.747  E  TAG: Hello — café Hello — café Round-trip encoding failed for encoding Western (Windows Latin 1)
21:15:36.747  E  TAG: s.data(using:) returned nil for encoding Western (ASCII)
21:15:36.757  E  TAG: After toLatin: éÅß中文, did: nil
21:15:36.758  E  TAG: localizedStandardCompare: 1
21:15:36.759  E  TAG: NSString compare: 1
21:15:36.761  E  TAG: true By default Swift strings treat them as canonical equiv
21:15:36.762  E  TAG: Decomposed: é, Precomposed: é
21:15:36.762  E  TAG: é é Decomposition gave something unexpected
21:15:36.762  E  TAG: é é Precomposition gave something unexpected
// testLocaleCurrentAndLocaleDependentProperties
    // The minimal variant supposedly only supports `en_001`.
    // This test checks what happens when you ask for a different locale.
    let localesToTest = ["fr_FR", "zh_Hant_TW", "de_DE", "en_US"]
    for id in localesToTest {
        let locale = Locale(identifier: id)
        // Access currency code & symbol
        let code = locale.currencyCode
        let symbol = locale.currencySymbol
        // Access a languageCode / regionCode
        let lang = locale.languageCode
        let reg = locale.regionCode
        
        print("Locale \(id): currencyCode = \(String(describing: code)), symbol = \(String(describing: symbol)), language = \(String(describing: lang)), region = \(String(describing: reg))")
    }
    
    // testTimeZoneLoading
    // Test a few zone identifiers, daylight savings, offset changes, etc.
    let tzIDs = ["America/New_York", "Europe/Paris", "Asia/Tokyo", "UTC"]
    for tzid in tzIDs {
        if let tz = TimeZone(identifier: tzid) {
            print("TimeZone \(tzid): secondsFromGMT = \(tz.secondsFromGMT()), abbreviation = \(String(describing: tz.abbreviation()))")
        } else {
            print("TimeZone(identifier: \"\(tzid)\") returned nil under minimal ICU data")
        }
    }
    
    // Also test nextDaylightTransition / isDaylightSavingTime etc
    if let ny = TimeZone(identifier: "America/New_York") {
        let now = Date()
        let isDST = ny.isDaylightSavingTime(for: now)
        print("NY is DST? \(isDST)")
        // This might return a default or be incorrect under minimal.
    }
    
    // testStringEncodingConversions
    let s = "Hello — café"  // includes non-ASCII
    // Try some encodings that require mapping tables
    let encodings: [String.Encoding] = [.isoLatin1, .windowsCP1252, .ascii]
    for enc in encodings {
        if let data = s.data(using: enc) {
            if let round = String(data: data, encoding: enc) {
                print(s, round, "Round-trip encoding failed for encoding \(enc)")
            } else {
                // decoding failed
                print("String(data:…, encoding:) returned nil for encoding \(enc)")
            }
        } else {
            print("s.data(using:) returned nil for encoding \(enc)")
        }
    }
    
    // testStringTransformTransliteration
    let mstr = NSMutableString(string: "éÅß中文")
    // Use e.g. “Latin-ASCII” transform
    let did = mstr.applyingTransform(.toLatin, reverse: true)
    //let did = mstr.applyTransform(.toLatin, reverse: false)
    print("After toLatin: \(mstr), did: \(did)")
    
    // testLocalizedStringCompareAndCollation
    let a = "straße"
    let b = "strasse"
    // Compare using locale-sensitive compare (e.g. German ß equivalence)
    let result = a.localizedStandardCompare(b)
    print("localizedStandardCompare: \(result.rawValue)")
    // Also test NSString compare with specific locale
    let nsA = a as NSString
    let nsB = b as NSString
    let comp = nsA.compare(nsB as String, options: [], range: NSRange(location: 0, length: nsA.length), locale: Locale(identifier: "de_DE"))
    print("NSString compare: \(comp.rawValue)")
        
    // testUnicodeNormalization
    let composed = "é"                 // U+00E9
    let decomposed = "e\u{0301}"       // 'e' + combining acute
    // Compare canonical equivalence
    print(composed == decomposed, "By default Swift strings treat them as canonical equiv")
    // Now test explicit normalization APIs
    let ns1 = (composed as NSString).decomposedStringWithCanonicalMapping
    let ns2 = (decomposed as NSString).precomposedStringWithCanonicalMapping
    print("Decomposed: \(ns1), Precomposed: \(ns2)")
    // In minimal ICU, these might not be implemented, perhaps returning the same or no change
    print(ns1, composed, "Decomposition gave something unexpected")
    print(ns2, decomposed, "Precomposition gave something unexpected")

KittyMac avatar Oct 08 '25 01:10 KittyMac

I got my larger project moved over to 6.2 and can confirm that lib_FoundationICU.so is indeed unstable when using the standard ICU data files. It sounds it is currently undocumented how to build the "special" icudata currently used in lib_FoundationICU.so?

KittyMac avatar Oct 09 '25 02:10 KittyMac

I got my larger project moved over to 6.2 and can confirm that lib_FoundationICU.so is indeed unstable when using the standard ICU data files.

Huge bummer. I was hoping that would be a way forward. Does "unstable" mean crashes, or weird behavior?

It sounds it is currently undocumented how to build the "special" icudata currently used in lib_FoundationICU.so?

The is a makefile in https://github.com/apple-oss-distributions/ICU, but as I mentioned at https://github.com/swiftlang/swift-foundation-icu/pull/53#issuecomment-2902270059, I wasn't ever able to get it to build. I may have been holding it wrong, though.

@iCharlesHu is the expert in that area, so maybe he has insights…

marcprux avatar Oct 09 '25 02:10 marcprux

Crashed in DateFormatter. I tried with the "minimal" and the larger "slim" (which just limits language to english) and they both crashed in the same spot.

Image

KittyMac avatar Oct 09 '25 02:10 KittyMac

I went back to my Android Swift 5.8 and rebuilt the icudata there using a slight variation on the "minimal" filter json - my libicudata.so size is now only 1.7 MB and initial tests with my project all work fine (no crashes).

I do like everything I've seen so far with the Swift 6.2 Android support; well done! If we figure out how to customize icudata in lib_FoundationICU.so then I think we'll update to it.

KittyMac avatar Oct 09 '25 20:10 KittyMac

Thanks for pointing out the upstream library that still isn't aligned, @MihaelIsaev, will try to fix it in an upcoming patch release.

finagolfin avatar Nov 08 '25 09:11 finagolfin