status-mobile icon indicating copy to clipboard operation
status-mobile copied to clipboard

ci: use arm64 macos hosts for iOS builds

Open jakubgs opened this issue 3 years ago • 14 comments

Lets see what happens...

jakubgs avatar Nov 25 '21 14:11 jakubgs

Of course:

[2021-11-25T14:35:28.383Z] No Android SDK tarballs are available for system architecture: aarch64-darwin

I need to merge https://github.com/status-im/status-react/pull/12797 first.

jakubgs avatar Nov 25 '21 14:11 jakubgs

Jenkins Builds

Click to see older builds (92)
:grey_question: Commit :hash: Finished (UTC) Duration Platform Result
:x: d208dd1d #1 2021-11-25 14:41:05 ~8 min ios :page_facing_up:log
:heavy_check_mark: d208dd1d #1 2021-11-25 14:46:44 ~14 min android :robot:apk :calling:
:heavy_check_mark: d208dd1d #1 2021-11-25 14:52:25 ~19 min android-e2e :robot:apk :calling:
:x: d6932e13 #2 2021-11-25 14:50:45 ~4 min ios :page_facing_up:log
:heavy_check_mark: d6932e13 #2 2021-11-25 14:59:50 ~13 min android :robot:apk :calling:
:heavy_check_mark: d6932e13 #2 2021-11-25 15:07:56 ~22 min android-e2e :robot:apk :calling:
:x: 7c26272c #3 2021-11-25 15:09:25 ~2 min ios :page_facing_up:log
:heavy_check_mark: 7c26272c #3 2021-11-25 15:27:02 ~19 min android :robot:apk :calling:
:heavy_check_mark: 7c26272c #3 2021-11-25 15:27:16 ~20 min android-e2e :robot:apk :calling:
:x: 65ebca96 #4 2021-11-25 16:50:55 ~3 min ios :page_facing_up:log
:heavy_check_mark: 65ebca96 #4 2021-11-25 17:02:42 ~15 min android :robot:apk :calling:
:x: aa190db0 #5 2021-11-29 15:39:30 ~3 min ios :page_facing_up:log
:heavy_check_mark: aa190db0 #5 2021-11-29 15:49:12 ~13 min android-e2e :robot:apk :calling:
:heavy_check_mark: aa190db0 #5 2021-11-29 15:53:08 ~17 min android :robot:apk :calling:
:heavy_check_mark: 8d9328a7 #6 2022-06-02 14:23:30 ~9 min android :robot:apk :calling:
:heavy_check_mark: 8d9328a7 #6 2022-06-02 14:25:37 ~11 min android-e2e :robot:apk :calling:
:heavy_check_mark: f7ca329d #7 2022-06-02 14:56:22 ~11 min android :robot:apk :calling:
:heavy_check_mark: f7ca329d #7 2022-06-02 14:56:39 ~11 min android-e2e :robot:apk :calling:
:x: b0739222 #8 2022-06-02 15:36:49 ~5 min ios :page_facing_up:log
:heavy_check_mark: b0739222 #8 2022-06-02 15:40:05 ~9 min android-e2e :robot:apk :calling:
:heavy_check_mark: b0739222 #8 2022-06-02 15:42:03 ~11 min android :robot:apk :calling:
:x: b2c9e449 #9 2022-06-02 15:46:47 ~4 min ios :page_facing_up:log
:heavy_check_mark: b2c9e449 #9 2022-06-02 15:53:33 ~11 min android-e2e :robot:apk :calling:
:heavy_check_mark: b2c9e449 #9 2022-06-02 15:53:46 ~11 min android :robot:apk :calling:
:x: adf4b4e4 #10 2022-06-03 09:39:47 ~4 min ios :page_facing_up:log
:heavy_check_mark: adf4b4e4 #10 2022-06-03 09:44:29 ~9 min android :robot:apk :calling:
:heavy_check_mark: adf4b4e4 #10 2022-06-03 09:51:16 ~16 min android-e2e :robot:apk :calling:
:x: 50ddc3da #11 2022-06-03 09:55:56 ~3 min ios :page_facing_up:log
:heavy_check_mark: 50ddc3da #11 2022-06-03 10:02:34 ~10 min android-e2e :robot:apk :calling:
:heavy_check_mark: 50ddc3da #11 2022-06-03 10:03:12 ~10 min android :robot:apk :calling:
:x: 50ddc3da #14 2022-06-03 11:03:24 ~2 min ios :page_facing_up:log
:heavy_check_mark: 50ddc3da #14 2022-06-03 11:13:03 ~12 min android-e2e :robot:apk :calling:
:heavy_check_mark: 50ddc3da #14 2022-06-03 11:13:07 ~12 min android :robot:apk :calling:
:heavy_check_mark: 9b9334b2 #12 2022-06-03 10:07:21 ~9 min android :robot:apk :calling:
:heavy_check_mark: 9b9334b2 #12 2022-06-03 10:08:18 ~10 min android-e2e :robot:apk :calling:
:x: 9b9334b2 #12 2022-06-03 10:09:39 ~11 min ios :page_facing_up:log
:heavy_check_mark: d217c5e4 #13 2022-06-03 10:42:45 ~11 min android-e2e :robot:apk :calling:
:heavy_check_mark: d217c5e4 #13 2022-06-03 10:42:59 ~11 min android :robot:apk :calling:
:heavy_check_mark: d217c5e4 #13 2022-06-03 10:52:42 ~21 min ios :iphone:ipa :calling:
:x: e6b791a3 #15 2022-06-09 09:18:45 ~6 min ios :page_facing_up:log
:heavy_check_mark: e6b791a3 #15 2022-06-09 09:24:03 ~11 min android-e2e :robot:apk :calling:
:heavy_check_mark: e6b791a3 #15 2022-06-09 09:24:07 ~12 min android :robot:apk :calling:
:x: d0116e0f #16 2022-06-09 09:21:53 ~2 min ios :page_facing_up:log
:heavy_check_mark: d0116e0f #16 2022-06-09 09:30:17 ~11 min android-e2e :robot:apk :calling:
:heavy_check_mark: d0116e0f #16 2022-06-09 09:32:01 ~12 min android :robot:apk :calling:
:x: 757eacf9 #17 2022-06-09 09:30:18 ~4 min ios :page_facing_up:log
:heavy_check_mark: 757eacf9 #17 2022-06-09 09:35:40 ~9 min android-e2e :robot:apk :calling:
:heavy_check_mark: 757eacf9 #17 2022-06-09 09:35:56 ~9 min android :robot:apk :calling:
:x: ec0eb680 #18 2022-06-09 10:40:01 ~2 min ios :page_facing_up:log
:heavy_check_mark: ec0eb680 #18 2022-06-09 10:46:51 ~9 min android-e2e :robot:apk :calling:
:heavy_check_mark: ec0eb680 #18 2022-06-09 10:47:05 ~9 min android :robot:apk :calling:
:x: 4b267af4 #19 2022-06-20 22:32:55 ~9 min ios :page_facing_up:log
:x: 4b267af4 #19 2022-06-20 22:34:55 ~3 min android :page_facing_up:log
:x: 4b267af4 #19 2022-06-20 22:35:32 ~4 min android-e2e :page_facing_up:log
:x: 4b267af4 #20 2022-06-20 22:39:28 ~8 min ios :page_facing_up:log
:x: d5d637a7 #21 2022-06-20 22:39:05 ~4 min ios :page_facing_up:log
:x: d5d637a7 #20 2022-06-20 22:39:40 ~5 min android :page_facing_up:log
:x: d5d637a7 #20 2022-06-20 22:42:53 ~8 min android-e2e :page_facing_up:log
:x: c56d4b38 #22 2022-06-20 23:27:10 ~5 min ios :page_facing_up:log
:x: c56d4b38 #21 2022-06-20 23:32:54 ~10 min android-e2e :page_facing_up:log
:x: c56d4b38 #21 2022-06-20 23:33:11 ~11 min android :page_facing_up:log
:x: 1dd6ccd0 #23 2022-06-21 11:18:18 ~2 min ios :page_facing_up:log
:heavy_check_mark: 1dd6ccd0 #22 2022-06-21 11:27:41 ~11 min android :robot:apk :calling:
:x: f38588a3 #26 2022-07-17 18:56:45 ~5 min ios :page_facing_up:log
:heavy_check_mark: f38588a3 #25 2022-07-17 19:00:49 ~9 min android :robot:apk :calling:
:heavy_check_mark: f38588a3 #25 2022-07-17 19:05:34 ~14 min android-e2e :robot:apk :calling:
:x: d9aa5a46 #27 2022-08-02 10:36:45 ~3 min ios :page_facing_up:log
:heavy_check_mark: d9aa5a46 #26 2022-08-02 10:42:30 ~9 min android-e2e :robot:apk :calling:
:heavy_check_mark: d9aa5a46 #26 2022-08-02 10:43:29 ~10 min android :robot:apk :calling:
:x: d9aa5a46 #1 2022-09-02 11:54:39 ~1 min ios :page_facing_up:log
:x: d9aa5a46 #1 2022-09-02 12:15:47 ~24 min android :page_facing_up:log
:heavy_check_mark: dfa016b7 #1 2022-09-12 09:21:19 ~2 min tests :page_facing_up:log
:heavy_check_mark: dfa016b7 #2 2022-09-12 09:27:05 ~7 min android :robot:apk :calling:
:x: dfa016b7 #2 2022-09-12 09:27:44 ~8 min ios :page_facing_up:log
:heavy_check_mark: dfa016b7 #2 2022-09-12 09:27:46 ~8 min android-e2e :robot:apk :calling:
:heavy_check_mark: a0794539 #2 2022-09-12 09:40:55 ~2 min tests :page_facing_up:log
:x: a0794539 #3 2022-09-12 09:41:24 ~2 min ios :page_facing_up:log
:heavy_check_mark: a0794539 #3 2022-09-12 09:46:21 ~7 min android-e2e :robot:apk :calling:
:heavy_check_mark: a0794539 #3 2022-09-12 09:46:38 ~7 min android :robot:apk :calling:
:x: a0794539 #4 2022-09-12 09:51:59 ~1 min ios :page_facing_up:log
:heavy_check_mark: bf27c036 #3 2022-09-12 10:42:47 ~8 min tests :page_facing_up:log
:heavy_check_mark: bf27c036 #4 2022-09-12 10:48:10 ~14 min android-e2e :robot:apk :calling:
:heavy_check_mark: bf27c036 #4 2022-09-12 10:49:28 ~15 min android :robot:apk :calling:
:x: bf27c036 #5 2022-09-12 10:52:26 ~18 min ios :page_facing_up:log
:heavy_check_mark: a0d26411 #4 2022-09-13 21:07:10 ~2 min tests :page_facing_up:log
:heavy_check_mark: a0d26411 #5 2022-09-13 21:12:36 ~7 min android-e2e :robot:apk :calling:
:heavy_check_mark: a0d26411 #5 2022-09-13 21:12:40 ~7 min android :robot:apk :calling:
:x: a0d26411 #6 2022-09-13 21:14:34 ~9 min ios :page_facing_up:log
:heavy_check_mark: 2849e2ac #5 2023-01-13 12:21:49 ~3 min tests :page_facing_up:log
:x: 2849e2ac #7 2023-01-13 12:24:56 ~6 min ios :page_facing_up:log
:heavy_check_mark: 2849e2ac #6 2023-01-13 12:27:20 ~8 min android-e2e :robot:apk :calling:
:heavy_check_mark: 2849e2ac #6 2023-01-13 12:29:32 ~10 min android :robot:apk :calling:
:grey_question: Commit :hash: Finished (UTC) Duration Platform Result
:heavy_check_mark: 35d6fa78 #8 2023-01-13 12:42:57 ~10 min ios :iphone:ipa :calling:
:heavy_check_mark: c9223cd9 #6 2023-01-13 12:46:56 ~2 min tests :page_facing_up:log
:heavy_check_mark: c9223cd9 #9 2023-01-13 12:52:32 ~7 min ios :iphone:ipa :calling:
:heavy_check_mark: c9223cd9 #7 2023-01-13 12:52:43 ~8 min android-e2e :robot:apk :calling:
:heavy_check_mark: c9223cd9 #7 2023-01-13 12:53:04 ~8 min android :robot:apk :calling:

status-im-auto avatar Nov 25 '21 14:11 status-im-auto

Looks like status-go build fails:

[2021-11-25T14:49:35.017Z] cgo: exec objc[61089]:: exec: "objc[61089]:": executable file not found in $PATH

jakubgs avatar Nov 25 '21 14:11 jakubgs

If I build locally it works:

[email protected]:~/workspace/status-react_prs_ios_PR-12849 % nix-build --no-out-link -A targets.status-go.mobile.ios default.nix
/nix/store/2paw9qzipvpxlc2ijlaiq3gi2g3c6g8z-status-go-0.91.5-1d752c0-ios

jakubgs avatar Nov 25 '21 15:11 jakubgs

Looks like the issue is indeed building of SQLCipher: https://github.com/status-im/status-react/issues/12799

16:39:00  Installing SQLCipher (3.4.2)
16:39:04  [!] /nix/store/gpf2hyxynkkijvqk2ijlw25jnb8q8fk4-bash-interactive-5.1-p8/bin/bash -c 
16:39:04  set -e
16:39:04  ./configure --enable-tempstore=yes --with-crypto-lib=commoncrypto CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_TEMP_STORE=2 -DSQLITE_SOUNDEX -DSQLITE_THREADSAFE -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_STAT3 -DSQLITE_ENABLE_STAT4 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_LOAD_EXTENSION -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS4_UNICODE61 -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_FTS5 -DHAVE_USLEEP=1 -DSQLITE_MAX_VARIABLE_NUMBER=99999"; make sqlite3.c
16:39:04  
16:39:04  checking build system type... 
16:39:04  Invalid configuration `arm64-apple-darwin20.6.0': machine `arm64-apple' not recognized
16:39:04  configure: error: /nix/store/ppzr8yab3s3883skd0da5i4ylzpksk9l-bash-5.1-p8/bin/bash ./config.sub arm64-apple-darwin20.6.0 failed

jakubgs avatar Nov 29 '21 15:11 jakubgs

@jakubgs still relevant ?

flexsurfer avatar Jan 03 '22 11:01 flexsurfer

Sure is, see https://github.com/status-im/status-react/issues/12799#issuecomment-988340136. Either we have to wait for Coreutils release with the fix, or we have to patch our own coreutils in nixpkgs. Or we have to get SQLite and then SQLCipher to update their config.sub and config.guess scripts. Another option is running everything via Rosetta binary translation in an x86_64 shell, but that kinda defeats the purpose of using an M1 machine in the first place...

jakubgs avatar Jan 03 '22 11:01 jakubgs

I tried applying just the patch to Coreutils version 9.0 but that fails, so it needs other changes:

src/uname.c:170:41: error: unknown type name 'MAYBE_UNUSED'
  170 | print_element_env (char const *element, MAYBE_UNUSED char const *envvar)
      |                                         ^~~~~~~~~~~~

So it cannot be easily patched.

jakubgs avatar Jan 03 '22 12:01 jakubgs

It appears iOS builds on M1 are failing with:

[2022-06-03T11:03:21.245Z] ❌  error: failed to read asset tags:
    The command `(
        cd /Users/jenkins/workspace/status-react_prs_ios_PR-12849/ios &&
            /Applications/Xcode.app/Contents/Developer/usr/bin/actool \
                --print-asset-tag-combinations \
                --output-format xml1 \
                /Users/jenkins/workspace/status-react_prs_ios_PR-12849/ios/StatusIm/Images.xcassets
    )`
exited with status 255. The command's standard error was:
[2022-06-03T11:03:21.245Z] 
[2022-06-03T11:03:21.245Z] 
[2022-06-03T11:03:21.245Z] ** ARCHIVE FAILED **
[2022-06-03T11:03:21.245Z] [12:03:20]: Exit status: 65

jakubgs avatar Jun 03 '22 13:06 jakubgs

Inside of /Users/jenkins/Library/Logs/gym/Status PR-StatusImPR.log there's this:

Backtrace:
  0   __exceptionPreprocess (in CoreFoundation)
  1   objc_exception_throw (in libobjc.A.dylib)
  2   -[DVTExtension valueForKey:error:] (in DVTFoundation)
  3   _IDEInitializeOnePlugInAndPrerequisites (in IDEFoundation)
  4   _IDEInitializePlugIns (in IDEFoundation)
  5   IDEInitialize (in IDEFoundation)
  6   IBIDEInitialize (in ibtoold)
  7   main (in ibtoold)
  8   start (in dyld)

Which is worrying.

jakubgs avatar Jun 03 '22 13:06 jakubgs

Some threads online suggest it's a bug in Xcode: https://stackoverflow.com/questions/50331031/xcode-archive-error-command-actool-failed-with-exit-code-255

While others say it's because of attribute slicing in Asset Catalog: https://developer.apple.com/forums/thread/13522

jakubgs avatar Jun 03 '22 13:06 jakubgs

This build on i7 Mac worked just fine:

  • https://ci.status.im/job/status-react/job/prs/job/ios/job/PR-12849/13/

But these failed:

  • Because of actool crashing without clear error
    • https://ci.status.im/job/status-react/job/prs/job/ios/job/PR-12849/12/
    • https://ci.status.im/job/status-react/job/prs/job/ios/job/PR-12849/17/
  • Because of status-go build error #13346
    • https://ci.status.im/job/status-react/job/prs/job/ios/job/PR-12849/14/
    • https://ci.status.im/job/status-react/job/prs/job/ios/job/PR-12849/15/
    • https://ci.status.im/job/status-react/job/prs/job/ios/job/PR-12849/16/

Seems unstable.

jakubgs avatar Jun 09 '22 09:06 jakubgs

Now we're seeing something new:

Building status-go for: ios,iossimulator
/nix/store/wqldq7k6f0am19x3qs7nar8w6gix3sq0-gomobile-unstable-2022-05-18/bin/gomobile: iossimulator/arm64: go build -ldflags -X github.com/status-im/status-go/params.GitCommit=22669d0423bc792e650150b01568870699ff1d25 -X github.com/status-im/status-go/params.Version=0.102.2 -s -w -buildmode=c-archive -o /private/tmp/nix-build-status-go-0.102.2-22669d0-ios.drv-0/gomobile-work/Statusgo-iossimulator-arm64.a ./gobind failed: exit status 2
# runtime/cgo
cgo: C compiler "objc[57723]:" not found: exec: "objc[57723]:": executable file not found in $PATH

EDIT: This is a known issue caused by xcrun warnings: https://github.com/status-im/status-mobile/issues/13949

jakubgs avatar Jun 20 '22 22:06 jakubgs

Currently the error appears to be:

Provisioning profile '94a83f39-326a-458c-b338-3a25675e8e5a' is not available on the Developer Portal for the user ****, fixing this now for you 🔨
...
[11:51:29]: $ xcodebuild -showBuildSettings -workspace ios/StatusIm.xcworkspace -scheme StatusImPR -configuration Release
[11:51:32]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
...
[!] xcodebuild -showBuildSettings timed out after 4 retries with a base timeout of 3. You can override the base timeout value with the environment variable FASTLANE_XCODEBUILD_SETTINGS_TIMEOUT, and the number of retries with the environment variable FASTLANE_XCODEBUILD_SETTINGS_RETRIES 

Some links that might be relevant:

  • https://github.com/fastlane/fastlane/issues/4621
  • https://developer.apple.com/forums/thread/95354
  • https://developer.apple.com/forums/thread/673778
  • https://stackoverflow.com/questions/55122513/

jakubgs avatar Sep 13 '22 21:09 jakubgs

Looks like the upgrade to Xcode in https://github.com/status-im/status-mobile/pull/14759 has fixed the issues. And the build takes only 8 minutes!

jakubgs avatar Jan 13 '23 12:01 jakubgs