Have a problem on Azure pipelines with Xcode 13.4 on macOS 12

[kvyatkovskys]

Description

Pipeline freezes when resolving swift packages

Virtual environments affected

• [ ] Ubuntu 18.04
• [ ] Ubuntu 20.04
• [ ] Ubuntu 22.04
• [ ] macOS 10.15
• [ ] macOS 11
• [X] macOS 12
• [ ] Windows Server 2019
• [ ] Windows Server 2022

Image version and build link

Environment: macOS 12 Version: 20220520.1

Is it regression?

no

Expected behavior

Pipeline should works correctly

Actual behavior

Pipeline freezes. Xcode 13.4 works fine on my MacBook.

Repro steps

• select a iOS project with swift packages
• run pipeline
• select macOS 12
• select Xcode 13.4

[miketimofeev]

Hi @kvyatkovskys! Could you prepare a workflow that can be used to reproduce the issue?

[kvyatkovskys]

Hi @kvyatkovskys! Could you prepare a workflow that can be used to reproduce the issue?

Hi @miketimofeev! You mean I need to provide more detailed steps 🤔

[miketimofeev]

@kvyatkovskys yes as we need to reproduce the issue to debug it further.

[kvyatkovskys]

@miketimofeev sure thing, will add later today 🙂

[kvyatkovskys]

env:

• macOS 12
• iOS project with swift packages
• fastlane (2.206.0) gym (to build project)

pipeline with settings below:

task Prepare environment:

• sudo xcode-select -switch /Applications/Xcode_13.4.app
• xcode-select --print-path
• gem install bundler:1.16.1
• bundle update --all

task Fastlane run dev:

• bundle exec fastlane build

steps:

• run pipeline
• select branch
• run

[kvyatkovskys]

any updates? 🙂

[kvyatkovskys]

@miketimofeev any updates?🙂

[miketimofeev]

@panticmilos did you have a chance to take a look?

[panticmilos]

Hi @kvyatkovskys,

Thank you for your patience, just to let you know I started investigating this issue in-depth today, so I will get back to you with some useful info as soon as possible.

Cheers

[kvyatkovskys]

Hi @kvyatkovskys,

Thank you for your patience, just to let you know I started investigating this issue in-depth today, so I will get back to you with some useful info as soon as possible.

Cheers

Thanks 🙂

[panticmilos]

Hi @kvyatkovskys,

Is there any chance you can provide us pipeline YAML file? I am aware of the screenshot you posted, but just to make sure I am not missing anything. Also if you could make public mock repo similar to yours that would be great(just to hold an app that I can use for the pipeline)

[kvyatkovskys]

Hi @panticmilos!

sure... pipeline works fine with Xcode 13.2.1

pool:
  name: Azure Pipelines
variables:
  XCODE_VERSION: '13.2.1'

steps:
- task: vs-publisher-473885.motz-mobile-buildtasks.ios-bundle-version.ios-bundle-version@1
  displayName: 'Bump iOS Versions in Example/Example-Info.plist'
  inputs:
    sourcePath: 'Example/Example-Info.plist'

- bash: |
   sudo xcode-select -switch /Applications/Xcode_$(XCODE_VERSION).app
   xcode-select --print-path
   gem install bundler:1.16.1
   bundle update --all
  displayName: 'Prepare environment '
  continueOnError: true
  env:
    GITHUB_ACCESS_TOKEN: $(GitHubAccessToken)

- script: 'bundle exec fastlane build'
  displayName: 'Fastlane run dev'
  env:
    GIT_AUTHORIZATION: $(GitHubPAT)
    SCHEME: Example
    CONFIGURATION: Debug
    MATCH_TYPE: adhoc
    EXPORT_METOD: ad-hoc
    APPLE_KEY_ID: $(APPLE_KEY_ID)
    APPLE_KEY_CONTENT: $(APPLE_KEY_CONTENT)
    APPLE_ISSUER_ID: $(APPLE_ISSUER_ID)
    OUTPUT_NAME: $(OUTPUT_NAME)
    EXPORT_OPTION: AdHoc
    IS_KEY_ENCODED: true
    DO_NOT_UPDATE_CERTIFICATES: true
    BUNDLE_ID: $(BUNDLE_ID)
    FORCE_UPDATE_PROFILES: false

- task: CopyFiles@2
  displayName: 'Copy Files to: $(build.artifactstagingdirectory)/Dev/'
  inputs:
    SourceFolder: '$(build.sourcesdirectory)'
    Contents: '$(OUTPUT_NAME).ipa'
    TargetFolder: '$(build.artifactstagingdirectory)/Dev/'

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: drop'
  inputs:
    PathtoPublish: '$(build.artifactstagingdirectory)'

[panticmilos]

hi @kvyatkovskys,

Just to check, is using this version, Xcode 13.2.1, a workaround you can use until we resolve this issue?

[kvyatkovskys]

Yes we can and we use Xcode 13.2.1. And we also plan to update Xcode

[kvyatkovskys]

Hi @panticmilos,

JFYI, the Xcode 13.4.1 also doesn't work too

[pavshr]

Hello, I have the same issue. I am trying to execute the following command xcodebuild -quiet build-for-testing -workspace MyApp.xcworkspace -scheme "MyAppScheme" -destination "name=iPhone 13 Pro" -derivedDataPath DerivedData in GitHub Actions.

Agent: macos-12 Xcode: 13.4.1

Everything works fine on Xcode 13.2

[kvyatkovskys]

Hi @panticmilos, any updates?

[kvyatkovskys]

still have problems

[panticmilos]

Hi @kvyatkovskys,

Is there any way you can provide me with a mock of the repository?

[kvyatkovskys]

Hi @kvyatkovskys,

Is there any way you can provide me with a mock of the repository?

hmm I don't know right now 🤔 this's a private repository... I noticed that when I removed the Twilio package from the project, everything works fine

[panticmilos]

@kvyatkovskys nice! Hope this will work for the others too. @pavshr, is removing the Twilio package applicable to your case as well?

[kvyatkovskys]

@kvyatkovskys nice! Hope this will work for the others too.

@pavshr, is removing the Twilio package applicable to your case as well?

Yeah 😀 but we use the Twilio package in our project. Still investigating the issue

[michaelmoneypenny]

@kvyatkovskys nice! Hope this will work for the others too. @pavshr, is removing the Twilio package applicable to your case as well?

Yeah 😀 but we use the Twilio package in our project. Still investigating the issue

I had an issue with twilio and bitcode, apparently disabling bitcode fixes the issue (bitcode is apparently deprecated)

[pavshr]

@panticmilos No, we are not using Twilio, and bitcode is also disabled for our app.

[kvyatkovskys]

I fixed the issue🙂. I added the twilio package as a binary xcframework to the project.

package.swift example

let package = Package(
    name: "VideoVisit",
    platforms: [.iOS(.v14), .macOS(.v10_15)],
    products: [
        // Products define the executables and libraries a package produces, and make them visible to other packages.
        .library(
            name: "VideoVisit",
            targets: ["VideoVisit", "TwilioVideo"]),
    ],
    dependencies: [
        // Dependencies declare other packages that this package depends on.
        .package(url: "https://github.com/SnapKit/SnapKit", .upToNextMajor(from: "5.0.0")),
    ],
    targets: [
        // Targets are the basic building blocks of a package. A target can define a module or a test suite.
        // Targets can depend on other targets in this package, and on products in packages this package depends on.
        .binaryTarget(
            name: "TwilioVideo",
            path: "TwilioVideo.xcframework"
        ),
        .target(
            name: "VideoVisit",
            dependencies: [
                "SnapKit",
                .target(name: "TwilioVideo"),
            ],
            path: "Sources",
            exclude: ["../VV-Example"],
            resources: [
                .process("../Sources/Media.xcassets")
            ]),
        .testTarget(
            name: "VideoVisitTests",
            dependencies: ["VideoVisit"]),
    ]
)

[kvyatkovskys]

I think, problem was due to a large file download. The twilio package is 60 mb in size.

cc @miketimofeev @panticmilos

import PackageDescription

let package = Package(
    name: "TwilioVideo",
    platforms: [
        .iOS("12.2")
    ],
    products: [
        .library(
            name: "TwilioVideo",
            targets: ["TwilioVideo"]),
    ],
    targets: [
        .binaryTarget(
            name: "TwilioVideo",
            url: "https://github.com/twilio/twilio-video-ios/releases/download/5.1.1/TwilioVideo.xcframework.zip",
            checksum: "125e944221600cbe98436d1cbc7bd8817c2795e6a9f36e3a63c01f0016f67d78"
        )
    ]
)

[jmprice18]

We've hit this issue on our side and we can reproduce it easily on MacOS 12 and Xcode 13.4+. We believe that Xcode is trying to access the keychain interactively while cloning at least 10 repositories from Github with a PAT. But if you only use public repositories, you won't run into this. We've tried using the PAT in git credential helper or via URL re-write and both produce the same error depending on the number of repositories xcodebuild is trying to resolve. The issue does not happen on XCode 13.2.1, even on the MacOS 12 agent.

Reproduce:

1. Use MacOS 12 and Xcode 13.4+.

2. Add a URL rewrite to gitconfig: git config --global url.https://username:[email protected] https://github.com

3. Run the pipeline and this will hang when cloning one of the repositories during xcodebuild.

Locally, we've tried to reproduce this error, and we can occasionally get a keychain prompt from XCode. See screenshot below on local dev machine.

@miketimofeev, any thoughts on how to resolve this?

[miketimofeev]

@jmprice18 we will take a look at what can be done from our side but at first glance, it looks like Xcode security limitations.

[panticmilos]

Hi @jmprice18, we were discussing how we could potentially overcome this issue, and for self-hosted runner, we could maybe run git config credential.helper cache after auth to avoid keychain prompts. In this case, we could avoid it. Of course, there is still a question to be resolved how we can avoid it for hosted runners.

[panticmilos]

Hi everyone. Since this looks like the security limitation of the Xcode and given the recent inactivity on the issue I am going to close it for now.

Feel free to continue the conversation here or reopen the issue. Cheers