actions
iOS test simulators fail to start on 20250811 macOS-15 build
Description
As of the 20250811 macos-15-arm64 release, Xcode appears to be unable to launch iOS simulators as part of an XCUnit test suite.
I have a XCUnit test suite that has worked for some time. It runs happily using Xcode 16.4 and an iOS 18.6 simulator on my own machine. It has also run successfully on every other iOS micro versions. It also ran on Xcode 16.0 and iOS 18.0 on the previous macos-15-arm64 image.
I am aware of #12541 and the changes to the included iOS SDK runtimes. I have applied the recommended change to explicitly select Xcode 16.4 in my actions workflow.
However, as of the macOS image change change, when I run my test suite, I get errors indicating that Xcode is unable to start an iOS simulator.
Platforms affected
- [ ] Azure DevOps
- [x] GitHub Actions - Standard Runners
- [ ] GitHub Actions - Larger Runners
Runner images affected
- [ ] Ubuntu 22.04
- [ ] Ubuntu 24.04
- [ ] macOS 13
- [ ] macOS 13 Arm64
- [ ] macOS 14
- [ ] macOS 14 Arm64
- [ ] macOS 15
- [x] macOS 15 Arm64
- [ ] Windows Server 2019
- [ ] Windows Server 2022
- [ ] Windows Server 2025
Image version and build link
The problem has been observed in:
- https://github.com/pypa/cibuildwheel (https://github.com/pypa/cibuildwheel/actions/runs/16973833037/job/48125995960)
- https://github.com/freakboy3742/pyspamsum/pull/75 (https://github.com/freakboy3742/pyspamsum/actions/runs/17034048860/job/48282410568?pr=75)
Is it regression?
Yes; any build against the previous macOS-15-arm64 image.
Expected behavior
It should be possible to run an Xcode project that starts an iOS simulator using:
xcodebuild test -project localtest/iOSTestbed.xcodeproj -scheme iOSTestbed -destination 'platform=iOS Simulator,name=iPhone 16e'
Actual behavior
The test project builds, but then fails to start the simulator, with an error:
Writing result bundle at path:
/private/var/folders/x7/ch5v91h56_zbvbd1y2f600dm0000gn/T/cibw-run-lfch6xkz/cp313-ios_arm64_iphonesimulator/testbed/20250818-071548.xcresult
2025-08-18 07:16:08.653 xcodebuild[1354:8026] [MT] IDELaunchReport: fbb6417437236880:fbb6417437235f00:Install Actions Finished with error: No matching device (A56B572A-E5CA-4A70-97B0-DF2686A8A2CF) in set at /Users/runner/Library/Developer/XCTestDevices
Domain: com.apple.CoreSimulator.SimError
Code: 404
--
2025-08-18 07:16:08.653 xcodebuild[1354:8026] [MT] IDELaunchReport: fbb6417437236880:fbb6417437235f00:Install Actions com.apple.dt.IDERunOperationWorkerFinished {
"operation_duration_ms" = 8;
"operation_errorCode" = 404;
"operation_errorDomain" = "com.apple.CoreSimulator.SimError";
"operation_name" = "_IDEInstalliPhoneSimulatorWorker";
}
2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 9.957 elapsed -- Testing started completed.
2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2025-08-18 07:16:16.226 xcodebuild[1354:8026] [MT] IDETestOperationsObserverDebug: 9.957 sec, +9.957 sec -- end
Testing failed:
iOSTestbed encountered an error (Failed to prepare device 'Clone 1 of iPhone 16e' for impending launch. (Underlying Error: No matching device (A56B572A-E5CA-4A70-97B0-DF2686A8A2CF) in set at /Users/runner/Library/Developer/XCTestDevices))
If I ssh into the machine and run the suite manually, I get the following error:
Writing result bundle at path:
/Users/runner/work/pyspamsum/pyspamsum/localtest/20250818-055037.xcresult
2025-08-18 05:50:49.068 xcodebuild[15556:60751] [MT] IDELaunchReport: 1ba0a4fe6dbd8740:1ba0a4fe6dbd8740: Finished with error: No matching device (E3CFACFD-04DA-48ED-92C4-0D763311D595) in set at /Users/runner/Library/Developer/XCTestDevices
Domain: com.apple.CoreSimulator.SimError
Code: 404
User Info: {
DVTErrorCreationDateKey = "2025-08-18 05:50:49 +0000";
IDERunOperationFailingWorker = "_IDEInstalliPhoneSimulatorWorker";
}
--
2025-08-18 05:50:49.069 xcodebuild[15556:60751] [MT] IDELaunchReport: 1ba0a4fe6dbd8740:1ba0a4fe6dbd8740: com.apple.dt.IDERunOperationWorkerFinished {
"device_identifier" = "FC1D56A6-910E-4DDA-8DC7-776480505F9C";
"device_model" = "iPhone17,5";
"device_osBuild" = "26.0 (23A5308g)";
"device_platform" = "com.apple.platform.iphonesimulator";
"device_thinningType" = "iPhone17,5";
"dvt_coredevice_version" = "477.29";
"dvt_coresimulator_version" = 1047;
"dvt_mobiledevice_version" = "1818.0.1";
"launchSession_schemeCommand" = Test;
"launchSession_state" = 1;
"launchSession_targetArch" = arm64;
"operation_duration_ms" = 11;
"operation_errorCode" = 404;
"operation_errorDomain" = "com.apple.CoreSimulator.SimError";
"operation_errorWorker" = "_IDEInstalliPhoneSimulatorWorker";
"operation_name" = IDERunOperationWorkerGroup;
"param_debugger_attachToExtensions" = 0;
"param_debugger_attachToXPC" = 0;
"param_debugger_type" = 1;
"param_destination_isProxy" = 0;
"param_destination_platform" = "com.apple.platform.iphonesimulator";
"param_diag_113575882_enable" = 0;
"param_diag_MainThreadChecker_stopOnIssue" = 0;
"param_diag_MallocStackLogging_enableDuringAttach" = 0;
"param_diag_MallocStackLogging_enableForXPC" = 0;
"param_diag_allowLocationSimulation" = 1;
"param_diag_checker_tpc_enable" = 0;
"param_diag_gpu_frameCapture_enable" = 3;
"param_diag_gpu_shaderValidation_enable" = 0;
"param_diag_gpu_validation_enable" = 1;
"param_diag_guardMalloc_enable" = 0;
"param_diag_memoryGraphOnResourceException" = 0;
"param_diag_mtc_enable" = 0;
"param_diag_queueDebugging_enable" = 1;
"param_diag_runtimeProfile_generate" = 1;
"param_diag_sanitizer_asan_enable" = 0;
"param_diag_sanitizer_tsan_enable" = 0;
"param_diag_sanitizer_tsan_stopOnIssue" = 0;
"param_diag_sanitizer_ubsan_enable" = 0;
"param_diag_sanitizer_ubsan_stopOnIssue" = 0;
"param_diag_showNonLocalizedStrings" = 0;
"param_diag_viewDebugging_enabled" = 0;
"param_diag_viewDebugging_insertDylibOnLaunch" = 0;
"param_install_style" = 2;
"param_launcher_UID" = 2;
"param_launcher_allowDeviceSensorReplayData" = 0;
"param_launcher_kind" = 0;
"param_launcher_style" = 0;
"param_launcher_substyle" = 0;
"param_runnable_appExtensionHostRunMode" = 0;
"param_runnable_productType" = "com.apple.product-type.application";
"param_structuredConsoleMode" = 0;
"param_testing_launchedForTesting" = 1;
"param_testing_suppressSimulatorApp" = 1;
"param_testing_usingCLI" = 0;
"sdk_canonicalName" = "iphonesimulator18.5";
"sdk_osVersion" = "18.5";
"sdk_variant" = iphonesimulator;
}
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 12.606 elapsed -- Testing started completed.
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2025-08-18 05:51:00.639 xcodebuild[15556:60751] [MT] IDETestOperationsObserverDebug: 12.606 sec, +12.606 sec -- end
Testing failed:
iOSTestbed encountered an error (Failed to prepare device 'Clone 1 of iPhone 16e' for impending launch. (Underlying Error: No matching device (E3CFACFD-04DA-48ED-92C4-0D763311D595) in set at /Users/runner/Library/Developer/XCTestDevices))
The only thing that stands out in this error log is the reference to Xcode 26; given than I'm explicitly using Xcode 16.4, it's not clear to me why iOS 26 would be involved.
Repro steps
The test suites that are affected are all using CPython's iOS testbed, by way of cibuildwheel.
The problem can be reproduced by downloading this tarball, unpacking it, then running:
$ curl -L https://github.com/beeware/Python-Apple-support/releases/download/3.13-b10/Python-3.13-iOS-support.b10.tar.gz -o Python-3.13-iOS-support.b10.tar.gz
$ tar zxvf Python-3.13-iOS-support.b10.tar.gz
$ python3.13 testbed clone localtest
$ python3.13 localtest run -- test test_platform
This will run a minimal subset of the Python test suite in an iPhone 16e simulator.
This sequence of commands works locally, but fails in the macos-15-arm64 environment as of the 20250811 update, regardless of what Xcode version has been set as the default.
localtest/__main__.py contains the code to launch Xcode and stream logs from the simulator that is created. The code starting xcodebuild is around L248 of that script.
Hi @freakboy3742 - Thank you for bringing this issue to our attention. We will look into this issue and will update you .
@archita105 this is directly connected to https://github.com/actions/runner-images/pull/12768.
This completely broke our CI.
Hey @gruselhaus!
The mentioned PR cannot affect the problem mentioned by the topic starter, since it belongs to a different version of the image (Intel-based macOS-15) and the changes have not yet been delivered.
Hey @freakboy3742!
The problem is either in the behavior of the beta simulators themselves, or in the way the build tool recognizes and sets targets. In this case, the build crashed before calling xcodebuild, judging by the general log. If the test suite recognizes Xcode 26 components as belonging to Xcode 16.4, then we need to understand how this happened. For our part, the most we can do is remove the beta runtimes from the image if this meets the needs of the community.
Hey @gruselhaus!
The mentioned PR cannot affect the problem mentioned by the topic starter, since it belongs to a different version of the image (Intel-based macOS-15) and the changes have not yet been delivered.
Yeah, sorry. You're absolutely right.
Hey @freakboy3742!
The problem is either in the behavior of the beta simulators themselves, or in the way the build tool recognizes and sets targets. In this case, the build crashed before calling
xcodebuild, judging by the general log. If the test suite recognizesXcode 26components as belonging toXcode 16.4, then we need to understand how this happened. For our part, the most we can do is remove the beta runtimes from the image if this meets the needs of the community.
We've added a work around to force a Xcode version
# -------------------------
# Setup Xcode (Hotfix, see: https://github.com/actions/runner-images/issues/12777)
# -------------------------
- name: Use Xcode 16.4
uses: maxim-lobanov/setup-xcode@v1
with:
xcode-version: "16.4"
- name: Download iOS 18 platform
run: |
sudo xcodebuild -license accept || true
sudo xcodebuild -downloadPlatform iOS
xcrun simctl list runtimes
We've added a work around to force a Xcode version
- name: Download iOS 18 platform
run: |
sudo xcodebuild -license accept || true
sudo xcodebuild -downloadPlatform iOS
xcrun simctl list runtimes
This is interesting. Basically, your code does exactly the same thing we do when building an image for each Xcode. If I understand correctly, this step should do nothing, since the simulators are already loaded and the license is accepted. But it fixes the problem, which is pleasing and surprising. Could you provide the build logs before and after applying the fix? 🙏 I would like to understand whether your problem is related to the stated topic starter and, if so, what exactly changed the behavior to the expected. 🤔
If you look at our build code, you can see that we do not interfere with the simulator settings and leave this to the standard methods provided by Apple - almost every attempt to improve something on our side leads to the fact that some of the features stop working 😄.
What I have noticed for me is when running jobs on a matrix, some runners are:
Image: macos-15-arm64
Version: 20250722.2025
And others are:
Image: macos-15-arm64
Version: 20250811.2170
Update: using macos-latest instead of macos-15 apparently helps and you always get 20250811.2170
We've added a work around to force a Xcode version
- name: Download iOS 18 platform run: | sudo xcodebuild -license accept || true sudo xcodebuild -downloadPlatform iOS xcrun simctl list runtimesThis is interesting. Basically, your code does exactly the same thing we do when building an image for each Xcode. If I understand correctly, this step should do nothing, since the simulators are already loaded and the license is accepted. But it fixes the problem, which is pleasing and surprising. Could you provide the build logs before and after applying the fix? 🙏 I would like to understand whether your problem is related to the stated topic starter and, if so, what exactly changed the behavior to the expected. 🤔
If you look at our build code, you can see that we do not interfere with the simulator settings and leave this to the standard methods provided by Apple - almost every attempt to improve something on our side leads to the fact that some of the features stop working 😄.
Sure, is there a way I can send you the build logs privately?
Hey @freakboy3742!
The problem is either in the behavior of the beta simulators themselves, or in the way the build tool recognizes and sets targets. In this case, the build crashed before calling
xcodebuild, judging by the general log. If the test suite recognizesXcode 26components as belonging toXcode 16.4, then we need to understand how this happened. For our part, the most we can do is remove the beta runtimes from the image if this meets the needs of the community.
@erik-bershel The build isn't crashing before calling xcodebuild - the errors are being raised by xcodebuild trying to start the test simulator.
I've spent another full day looking into this. To me, all signs point to something being fundamentally different in the behavior of the simulators.
This commit includes a test pass that invokes xcodebuild directly, in verbose mode. I've run this 5 times; four of those times, it completed, with runtimes ranging between 2'25" and 8'04". That spread is fairly concerning by itself; but one of the five times, it didn't complete after 40 minutes, providing no console output. This isn't behavior I've ever seen locally.
So - it's not just xcodebuild - but the simulators are highly variable in performance, and sometimes fail in highly undesirable ways.
I'm not just running xcodebuild by itself, though. I'm trying to get the contents of stdout while the simulator runs, so the testbed script starts 2 processes - xcodebuild (doing the same command line as the the successful task), plus a second process that polls waiting for the simulator to start (using xcrun simctl --set testing list -j to identify the UDID of a freshly booted simulator, then xcrun simctl --set testing spawn <udid> log stream to display the simulator's logs.
You can see the result of running this script in this commit, with this buildlog.
As best as I can make out, it never successfully detects a running simulator - xcodebuild gets to the point where it should start a test simulator; but then fails with the "No matching device in set" error.
This testbed script is currently working successfully on macos-13 x86_64 images, and on macos-14 ARM64 images; and up until the 20250811 release, ran successfully on macos-15 ARM64 images as well. There's copy of the testbed runner script (with some additional debug statements showing what is being executed) here
On thing that did work was reverting to Xcode 16.0 - I had to do the "install under Xcode 16.4, then switch to Xcode 16.0" trick (see this commit); the build log for that commit also fails; but interestingly, the calls to list devices initially fails because the device list directory (/Users/runner/Library/Developer/XCTestDevices) doesn't exist - something that isn't true of builds on Xcode 16.4. So - this worked, but it's obviously a less than optimal solution, given that (a) it requires a fairly time-expensive download to make it work, and (b) the fact that it requires reverting to a quite old Xcode version (and the version that will no longer be the default as of next week) doesn't bode well for switching to Xcode 26 as a default in what I presume will eventually be a macos-26 image.
I've also tried the proposed solution of explicitly installing (or re-installing the iOS 18.5 simulator) - see this commit; that doesn't fix anything (build log).
I'm running out of ideas for how to diagnose this any further. I can't reproduce this failure locally on my machine, and the only way I can get access to the Github Actions machine where it manifests is to use tools like mxschmitt/action-tmate, which provides a very limited window through which to diagnose anything.
To me - all signs point to something in the installed simulator images being different in fairly substantial way.
Hey @freakboy3742! 👋
You did a really 🆒 job investigating the problem. I'm sorry that these changes took so much of your time. Both machine and personal. 😞
While we investigate what's happening (and it's not entirely clear to me yet why the behavior is exactly like this - there was no such strange glitch before, but beta is always beta), I suggest you stay on macos-14 if possible (saw your changes), 99 out of 100 that the problem will resolve itself in one of two cases: after we change the default version on our side or after the release of Xcode 26. Until then, we will not sit idly by, but will try to fix the situation on our side. This is a really nasty degradation, but I can't provide a solution yet, other than uninstalling Xcode 26 beta, which doesn't meet the community's needs, or trying to uninstall beta simulators, which is also not very cool, because for those who suffered from incompatibility there will be an option to move to macos-14 relatively painlessly, but for beta users there will be no such workaround.
A status update - I've been able to work around the problem by restructuring how we're tracking logs from the iOS simulator.
This avoids the immediate "crash on run" problem for us; however, we're still seeing highly variable launch times associated with simulators. This PR has a bunch of runtime experiments; showing runtimes between 2:37 and 4:58.
The actual test in those experiments is minimal - the test itself takes about 2 seconds to run, once the simulator has launched. Compiling the test project takes about a minute. That means starting a simulator is taking between 1:30 and 4:00. That's a pretty big performance spread, but it's on the edge of workable (although there's a definite echo of previous performance regressions, like #9591 and #11509).
However there's also one example out of 11 which is an apparent "failure to start" of the simulator. I manually stopped the run after 26 minutes - 25 minutes seems more than enough time for a simulator to start. That's a disturbing failure rate - I reported a similar failure in my earlier test results.
If you look at our build code, you can see that we do not interfere with the simulator settings and leave this to the standard methods provided by Apple - almost every attempt to improve something on our side leads to the fact that some of the features stop working 😄.
Interesting that these are pwsh scripts and not shell scripts. Wonder if that will make a difference since most/all of the rest of the installers are shell.
I'm aware that GitHub runners also use Anka, and I've had some similar issues setting up self hosted Anka runners on prem. There is additional requirements for the host machine to be setup correctly for the child VMs to run and setup xcode correctly.
(**) ARM USERS: Creation of 15.3.x VMs on a 15.x host currently requires you to prepare the host. The following steps are required:
- Install Xcode 16.2 or later and set it up fully with the following commands:
XCODE_DESTINATION="/Applications"
XCODE_APP="Xcode.app"
sudo /usr/sbin/dseditgroup -o edit -a everyone -t group _developer
sudo xcode-select -s ${XCODE_DESTINATION}/${XCODE_APP}/Contents/Developer
sudo xcodebuild -license accept
sudo xcodebuild -runFirstLaunch
sudo DevToolsSecurity -enable
for PKG in $(/bin/ls ${XCODE_DESTINATION}/${XCODE_APP}/Contents/Resources/Packages/*.pkg); do
sudo /usr/sbin/installer -pkg "$PKG" -target /
done
echo "Checking Xcode CLI tools"
sudo xcode-select -s "${XCODE_DESTINATION}/${XCODE_APP}"
xcode-select -p &> /dev/null
if [ $? -ne 0 ]; then
echo "Xcode CLI tools not found. Installing them..."
touch /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress;
PROD=$(softwareupdate -l |
grep "\*.*Command Line" |
tail -n 1 | sed 's/^[^C]* //')
softwareupdate -i "$PROD" --verbose;
else
echo "Xcode CLI tools OK"
fi
- Install the device support package.
Then on the child VM:
(**) ARM USERS: When creating 15.x VMs on 14.x hosts, you must ensure that Xcode 16.2 OR the MobileDevice.pkg (inside of Xcode.app) is installed.
ARM USERS: There is also a rare problem where your Xcode is not fully set up and still creates problems. Be sure to run the following:
sudo xcodebuild -license accept
sudo xcodebuild -runFirstLaunch
for PKG in $(/bin/ls /Applications/Xcode.app/Contents/Resources/Packages/*.pkg); do
sudo /usr/sbin/installer -pkg "$PKG" -target /
done
Taking a quick look though the build scripts I didn't see the mobile device package installs being run.
Also think this is the main reason I opened https://github.com/actions/runner-images/issues/12961
This is still happening for us (Xcode 26 and iOS 26 sim) - despite those workarounds
Hey @theblixguy!
Please clarify which specific image you're talking about, preferably with the version. And it would be great if you could share a snippet of the code you're using as a workaround so we can better understand the situation.
Hey @theblixguy!
Please clarify which specific image you're talking about, preferably with the version. And it would be great if you could share a snippet of the code you're using as a workaround so we can better understand the situation.
The most recent one but it doesn't matter as it's been happening for several weeks now even on the previous image versions.
The workaround I'm referring to here is downloading the simulator runtime as suggested in some other threads.
Hey @erik-bershel! Is there any update on this issue? We are also facing the same behavior with simulator stuck when booting. This behavior has been the same starting from runners with first xcode 26.0 beta and remains the same. Right now we are on runner 20250922.2357 and running with Xcode 26.0, we are facing it only with sim iOS 26.0. With iOS 18.5 works fine for us. Also tried to use workaround mentioned with warming up via printing device list but it didn't help.
Hey @asos-oleksandrmyronovych!
Users helped isolate one of the sources of the issue here, but there's no guarantee it will lead to a complete resolution. After updating the images this week, I'll conduct additional resolution testing and come back with an update. I hope it'll be a fix, but there's a chance it's a result of the Core SimRuntime update that arrived with Xcode 26.
@erik-bershel heya! any update on the issue? I tried running with latest runner (20250928.2397) and it still has problem with booting iOS 26 sim
@erik-bershel troubleshooting simulator boot hanging and overall longer runs after updating workflows to xcode26 runners, were you able to confirm if #13149 fixed the simulators? Reckon it'll only be deployed in the next image update?
Heya! 👋
any update on the issue? I tried running with latest runner (20250928.2397) and it still has problem with booting iOS 26 sim
Lost in logs. Digging deeper and deeper here. Seems like Xcode 26 and latest system updates making it harder to get the same speed from the same hardware that we have. I'm seeing a noticeable performance degradation compared to earlier runs using Xcode 16.4, but I can't yet pinpoint a clear cause. Unfortunately, the system logs don't show any obvious, recurring errors, and synthetic performance tests don't reflect real-world scenarios. Worst of all, the issues are sporadic, even with completely identical setups and resulting logs.
The first suspect is MDS, as its actions create noticeable queues when working with the disk, but I can neither disable it nor recommend such an action, as this puts an end to most of the functionality of macOS.
I'm also working on optimization attempts. Simulator warmups and the like, but so far there haven't been any visible improvements. I stumbled across a decent solution, but we can't implement it technically—the caches take up too much disk space. 😞
@erik-bershel troubleshooting simulator boot hanging and overall longer runs after updating workflows to xcode26 runners, were you able to confirm if https://github.com/actions/runner-images/pull/13149 fixed the simulators?
Yes and no. We've run several tests on real projects that have shown improvement, but some sporadic performance remains, which suggests there's some additional bottleneck, as I mentioned above. For example, for one of these tests, the expected build and test duration is 15 +/- 1 minute. The actual duration is now between 15 and 20 minutes; before implementation, the range was between 15 and 35 minutes. However, some tests still show a range from a few minutes to timeouts. The investigation is ongoing.
UPD:
Reckon it'll only be deployed in the next image update?
Yup! Next week presumably.
@erik-bershel thank you for the update and your effort to solve this!
For example, for one of these tests, the expected build and test duration is 15 +/- 1 minute. The actual duration is now between 15 and 20 minutes; before implementation, the range was between 15 and 35 minutes.
Observed the same, throwing an hammer to OS services (specially MDS) seems to have helped stabilise duration for now. Building is still slower than it used to be, we just have less random run duration swings of additional 10-20 minutes.
However, some tests still show a range from a few minutes to timeouts.
About the timeouts and as an additional data point, after splitting the whole process into independent steps for (1) booting simulator, (2) build and (3) test (in that order) we're only observing occasional timeouts in the (1) booting simulator step. When the simulator boots correctly, everything else runs smoothly. The combination of shutting off OS services and splitting these steps also reduced the percentage of runs that timeout due to the simulator hanging, but without additional AB tests that may have been just a happy coincidence.
Building is still slower than it used to be
Yes, that's true. It was the result of a chain of events. Xcode 26.0 RC and newer required macOS 15.6 system update, macOS 15.6 required a hypervisor update, and the hypervisor update resulted in the use of a slower disk interaction model. However, this change alone shouldn't have caused the crashes, as this slower option was used from the very beginning and was replaced by a faster one relatively recently.
Frankly, it seems the instability is the result of a combination of factors. The updated simulators are more resource-intensive, plus the rollback to a slower disk access model has led to some Xcode- and simulator-related services and processes periodically failing to respond within the allotted time and crashing or freezing.
@erik-bershel Really appreciate the detail, knowing that helps confirm I'm not crazy :D
Splitting the simulator booting step further, here's a run with a timeout vs successful run. It's xcrun simctl bootstatus that's hanging without printing any logs. When it does work, there's always a data migration process. While I'm not sure if this data migration is responsible for the timeouts, could the runner install process handle the data migration to speed up the simulator boot process?
Sidenote, wonder if brute forcing the simulator to shutdown and restart would workaround the timeouts 🤔
Edit: using xcrun simctl bootstatus <device> -b shows additional logs, it's hanging "Waiting on BackBoard". Run.
Hey @vvolkgang!
I've previously tried to package the simulator warmup process into an image build. But we ran into a problem. 😅 Warming up all the simulators for three platforms (the minimum specified in the policy) requires just over 40 GB of disk space for caches and a little over eight hours, which exceeds the build time limit.
UPD: Oh, and it makes the build process completely unpredictable. Plus, I'm not sure how these simulators will behave in production, especially after switching Xcode versions.
Hey
I'm also experiencing issues with running parallel tests via xcodebuild on Xcode 16.3 after installing Xcode 26.0.1 at our CI: No matching device (UDID) in set at /Users/runner/Library/Developer/XCTestDevices
I've found that Xcode 26.0.1 replaces files in /Library/Developer/PrivateFrameworks/CoreSimulator.framework, which breaks tests on Xcode 16.3.
I'm testing this workaround:
- I saved two backups of /Library/Developer/PrivateFrameworks/CoreSimulator.framework:
16.3-CoreSimulator.frameworkfrom Xcode 16.3 (before installing Xcode 26.0.1)26.0.1-CoreSimulator.frameworkfrom Xcode 26.0.1 (after installing Xcode 26.0.1)
- In the jobs, I replace the entire contents of /Library/Developer/PrivateFrameworks/CoreSimulator.framework with the appropriate version of the framework. If the job uses:
- Xcode 26.0.1 -> uses artifacts from
26.0.1-CoreSimulator.framework - Xcode 16.3 -> uses artifacts from
16.3-CoreSimulator.framework
- Xcode 26.0.1 -> uses artifacts from
This approach seems to fix the main problem:
- parallel tests on Xcode 16.3 remain working
- it's possible to use Xcode 26.0.1
The downsides (that I've noticed so far):
- I see various errors in the logs, but they don't seem to affect the job results (maybe they can be adjusted)
- it's possible this will affect something else that I haven't encountered yet
Maybe this observations will help you?
@freakboy3742 you're probably just missing (1) pinning an xcode version and (2) parsing the UDID of a matching installed simulator. Here's an example https://github.com/bitwarden/ios/blob/c76326b3e43147befd4380db80c18fe26ded4acd/.github/workflows/test.yml#L128-L130
@vvolkgang I don't think that's the problem - I'm doing both those things. If I select an iPhone 16e running iOS 18.5, the simulator starts promptly; if I select the same simulator, but running iOS 26, it doesn't start.
@erik-bershel Seems like Apple acknowledged simulators hanging in macOS VMs is a known issue and is working on it https://discuss.circleci.com/t/xcode-26-rc/54066/21
Warming up all the simulators for three platforms (the minimum specified in the policy) requires just over 40 GB of disk space for caches and a little over eight hours, which exceeds the build time limit.
Perhaps this could be provided for a single simulator image per xcode version, as that would speed up the most common testing scenario.
edit: for the simulator hangs retrying seems to be working :finnadie: https://github.com/bitwarden/ios/actions/runs/18710405334/job/53357269720