Cross compilation for amazonlinux fails on macOS

[matrejek]

Describe the bug I am including aws-sdk-swift aversion 5.0.0-alpha.6 as a dependency to my package. While compiling from Xcode - it works fine. While cross-compiling using below command

swift build --destination /usr/local..../swift-5.3-amazonlinux2.xtoolchain/destination.json

build fails on importing Crypto.

I think that it might be caused by os directive used below as os() for cross-compilation will return value not by the target platform but for the one used for building. Therefore it returns false on macOS, so useSwiftCrypto = false and dependency below is not added at the end.

// switch for whether to use swift crypto. Swift crypto requires macOS10.15 or iOS13.I'd rather not pass this requirement on
#if os(Linux)
let useSwiftCrypto = true
#else
let useSwiftCrypto = false
#endif

// Use Swift cypto on Linux.
if useSwiftCrypto {
    package.dependencies.append(.package(url: "https://github.com/apple/swift-crypto.git", from: "1.0.0"))
    package.targets.first { $0.name == "AWSCrypto" }?.dependencies.append(.product(name: "Crypto", package: "swift-crypto"))
}

To Reproduce Steps to reproduce the behavior:

1. Include as a package dependency aws-sdk-swift 5.0.0-alpha.6
2. Build the package using the command above

Actual behavior Build fails on importing Crypto

Expected behavior Build works

Setup (please complete the following information):

• OS: macOS, Xcode 12 beta 6
• Version of aws-sdk-swift 5.0.0-alpha.6
• Authentication mechanism: n/a

Additional context n/a

[adam-fowler]

Ok. We need the switch on os(Linux) to ensure we aren't including Crypto in the macOS/iOS builds. On appleOS builds Crypto requries a minimum platform of iOS 13 and macOS 10.14, I would rather not pass that requirement onto clients of aws-sdk-swift.

I don't have a great deal of experience with the cross compilation build architecture. Is there something else we can test during cross compilation builds? An environment variable perhaps?

Also Swift 5.3 will come with platform conditions eg

.product(name: "Crypto", condition: .when(platforms: [.linux])),

These may work with cross compilation builds, I don't know. I've just asked this question in the swiftpm slack.

[adam-fowler]

@matrejek I got no joy from the swiftpm slack. Would you be able to test this for me seeing as you have cross compilation setup already. Add

.package(url: "https://github.com/apple/swift-crypto.git", from: "1.0.0") 

to the list Package.dependencies and add

.product(name: "Crypto", condition: .when(platforms: [.linux]))

as a dependency of "AWSCrypto"

thanks

[matrejek]

@adam-fowler Sure, I will check it today or at the latest tomorrow as I should get my MacBook back from service today.

[matrejek]

Hi, @adam-fowler, sorry to keep you waiting but getting back to my work setup took me a while yesterday.

So, basically I have performed the following changes:

// swift-tools-version:5.1

to

// swift-tools-version:5.3

as .product with condition comes from 5.3.

Added

.package(url: "https://github.com/apple/swift-crypto.git", from: "1.0.0") 

to the dependencies list and

 .product(name: "Crypto", package: "swift-crypto", condition: .when(platforms: [.linux]))

And now it gets past compiling the file that used to fail before, so I guess that solves the issue with Crypto but now it fails later on

/Users/mateusz/Downloads/spm-playground/aws-sdk-swift-core/Sources/AWSXML/Expat.swift:64:21: error: cannot find 'strlen' in scope
        let cslen = strlen(cs) // cs? checks for a NULL C string
                    ^~~~~~

I guess it may be worth to create a separate issue for tracing that problem. Probalby strlen is somehow unavailable on linux for Swift?

[adam-fowler]

That strlen thing is a little weird. I have CI compiling on Linux Swift 5.2 and the nightly builds of Swift 5.3 without issue. Can you create a separate issue for this. Might be another cross compilation issue

[adam-fowler]

Can you try adding import SwiftShims at the top of Expat.swift and see what that does.

[matrejek]

It worked but leads to linker failure. This is where my cross-compilation knowledge ends.

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: _length_code in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(trees.o)
>>> referenced by deflate.o:(deflate_slow) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: _dist_code in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(trees.o)
>>> referenced by deflate.o:(deflate_slow) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: _dist_code in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(trees.o)
>>> referenced by deflate.o:(deflate_slow) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: _length_code in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(trees.o)
>>> referenced by deflate.o:(deflate_fast) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: _dist_code in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(trees.o)
>>> referenced by deflate.o:(deflate_fast) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: _dist_code in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(trees.o)
>>> referenced by deflate.o:(deflate_fast) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
eset) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflateReset) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflateReset) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflate) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: _length_code in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(trees.o)
>>> referenced by deflate.o:(deflate) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflateParams) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflateParams) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
onlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflateParams) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflateParams) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflateParams) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(deflateParams) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: zcfree in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(zutil.o)
>>> referenced by deflate.o:(deflateInit2_) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_32S against symbol: zcalloc in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(zutil.o)
>>> referenced by deflate.o:(deflateInit2_) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: can't create dynamic relocation R_X86_64_64 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output
>>> defined in /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a(deflate.o)
>>> referenced by deflate.o:(configuration_table) in archive /usr/local/lib/swift/dst/x86_64-unknown-linux/swift-5.3-amazonlinux2.xtoolchain/x86_64-amazonlinux2.sdk/usr/lib/gcc/x86_64-redhat-linux/7/../../../../lib64/libz.a

ld.lld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

[adam-fowler]

I can probably rewrite the code not to use strlen thus avoid this issue completely

[adam-fowler]

One other thing that might be useful to find out. Does it compile when cross compiling for Ubuntu?

[adam-fowler]

Oh and one other thing you could try is replace the

import SwiftShims

with

#if os(Linux)
import Glibc
#else
import Darwin.C
#endif

[MoridinBG]

@matrejek for the linker issue - update your amazonlinux destination. I encountered the libz issue yesterday - it was missing in the SDK and it got fixed - https://github.com/SPMDestinations/homebrew-tap/issues/9

Unfortunately I encountered another issue when cross compiling the stable (4.7.0) aws-sdk-swift-core for amazonlinux2: .build/checkouts/aws-sdk-swift-core/Sources/AWSSDKSwiftCore/XML.swift:13:8: error: missing required module 'CFXMLInterface' Building in amazonlinux2 Docker image works fine, so it's not a Linux issue. I am chasing what is missing from the SDK now.

[adam-fowler]

@MoridinBG v4.7.0 is dependent on FoundationXML this dependency no longer exists in v5. Can you try your tests on the 5.0.0-alpha.6 release.

[MoridinBG]

With the latest alpha tag when cross compiling I get

TestSDK/.build/checkouts/aws-sdk-swift-core/Sources/AWSCrypto/exports.swift:18:19: error: no such module 'Crypto'
@_exported import Crypto`

This on an empty swift package init --type executable that only imports the sdk core as a dependency. Builds successfully in the Docker image, so it's again something with the cross-compilation SDK. Some other missing library? libssl, etc? Does FoundationXML in 4.7.0 rely on libxml that might be missing from the SDK too?

The SDK + toolchain is from here. It is used in conjunction with the swift build --destination flag

[matrejek]

@MoridinBG thanks! updating the destination worked.

So, summarizing - after changes from 9 Sep and adding

#if os(Linux)
import Glibc
#else
import Darwin.C
#endif

in the top of Sources/AWSXML/Expat.swift it compiles fine for amazonlinux :)

[MoridinBG]

Is it getting confused by being compiled on Darwin, but for Linux? Shouldn't it be targeting Linux from the package point of view?

[matrejek]

@MoridinBG I think that we might have solved that crypto error above.

[MoridinBG]

🤦‍♂️yeah, just noticed that... Interesting that when building natively in the container you do not need to add it as an extra dependency, so it must be something with the toolchain.

[matrejek]

@adam-fowler are we fine with bumping to 5.3? Then I can create PR to fix that.

[adam-fowler]

There is a PR for 5.3 already #379 . Unfortunately it seems like github actions are still using 5.2 on macOS. I would rather wait until we can have macOS CI working before we merge

[adam-fowler]

@matrejek Do you want to verify that works with cross compilation?

[matrejek]

Sorry, I have missed you comment. Will try to check them at the beginning of this week.

[adam-fowler]

@matrejek just realised the dependency checking when using the command line is broken (works in Xcode). If I run swift build on the upgrade-to-swift-5.3 branch I get the following error

error: the library 'swift-5.3-deps' requires macos 10.10, but depends on the product 'Crypto' which requires macos 10.15; consider changing the library 'swift-5.3-deps' to require macos 10.15 or later, or the product 'Crypto' to require macos 10.10 or earlier.

I've added a bug https://bugs.swift.org/browse/SR-13761

[adam-fowler]

@matrejek Given we can't use the swift 5.3 condition: .when(platforms: [.linux]) what are your thoughts on this #403 as a solution. It is not ideal but would allow us to specify when we are doing cross compilation.

[matrejek]

It's hacky but seems legit. Actually, I think that cross-compilation may be a kind of true edge case as I guess the majority use docker for building.

Btw, sorry for late response - happened to have my machine broken again so I have limited ability to play with stuff.

[adam-fowler]

This is no longer an issue in the 7.0 alpha