Simulator not lauching on CI workflow with Expo

[brpinto]

Description

Hello there,

I am working on a mobile application with expo SDK 50, [email protected] and I am using Detox for my e2e tests. It used to work well on Github Actions, but it seems that my action these days has trouble to find a simulator to run the tests.

NB: tests run perfectly locally, the issue occurs on CI only.

  DetoxRuntimeError: Failed to find a device by type = "iPhone 15 Pro Max"

    HINT: Run 'applesimutils --list' to list your supported devices. It is advised only to specify a device type, e.g., "iPhone Xʀ" and avoid explicit search by OS version.

I took a look to the react-native example on wix/detox repo to have an idea of the right configuration for my .detoxrc.js and ended to this chunk for iOS sim :

devices: {
  simulator: {
    type: 'ios.simulator',
    device: {
      type: 'iPhone 15 Pro Max',
      os: '17.2'
    },
  },
},

My yml file is as code below. I am not giving any configuration for the simulator, but I don't know if it's needed as long as I never saw something like that on Detox documentation or examples.

name: e2e-ios
on:
  push:
    branches:
      - main
      - 'SPA-**'
      - 'FEATURE-**'

jobs:
  e2e-ios:
    runs-on: macos-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3

      - name: Install Yarn dependencies
        run: yarn --frozen-lockfile --prefer-offline

      - name: Install macOS dependencies
        run: |
          brew tap wix/brew
          brew install applesimutils
        env:
          HOMEBREW_NO_AUTO_UPDATE: 1
          HOMEBREW_NO_INSTALL_CLEANUP: 1

      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.2
          bundler-cache: true

      - name: Expo pre-build
        run: npx expo prebuild -p ios

      - name: Install Detox
        run: |
          yarn global add detox-cli
          yarn add detox

      - name: Detox rebuild framework cache
        run: yarn detox rebuild-framework-cache

      # - name: Cache Detox build # needed ?
      #   id: cache-detox-build
      #   uses: actions/cache@v3
      #   with:
      #     path: ios/build # WARNING : right here ??
      #     key: ${{ runner.os }}-detox-build
      #     restore-keys: |
      #       ${{ runner.os }}-detox-build

      - name: Detox build
        run: MY_APP_MODE=mocked detox build -c ios.sim.release

      - name: Detox test
        run: MY_APP_MODE=mocked detox test -c ios.sim.release --cleanup

      - name: Upload artifacts # what are those ?
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: detox-artifacts
          path: artifacts

Maybe I am missing something obvious here, but please tell me if you need more information or details about my code.

Thank your for your help, I'll be pleased to discuss further with you on this subject.

Your environment

Detox version: 20.13.5 React Native version: 0.73.3 Node version: 20.7.0 Device model: Macbook Air M1, 2020 OS: Sonoma 14.0 Test-runner (select one): jest

[ashermja]

Hi @brpinto, We were facing the same issue. Sees there is an issue with the new default image which is macos-ventura-13.6-xcode-15.2 probably related to xcode 15.2 and detox incompatibility? To get round the issue we specified macos-ventura-13.4-xcode-14.3.1 in eas.json i.e.

"ios": {
        "simulator": true,
        "image": "macos-ventura-13.4-xcode-14.3.1",
        "cocoapods": "1.14.3"
},

Note in our particular situation we had to specify cocoapods": "1.14.3" as the new version of one of the pods was incompatible with the 14.3.1 image

[karlhorky]

We're also experiencing something similar on GitHub Actions, with the macos-13 runner:

Fails on iOS Detox tests with:

pnpm detox test --configuration ios
11:25:07.073 detox[23852] B jest --config ./e2e/jest.config.cjs e2e
FAIL e2e/appStarts.e2e.ts
  ● Test suite failed to run

    DetoxRuntimeError: Failed to find a device by type = "iPhone 13 Pro"

    HINT: Run 'applesimutils --list' to list your supported devices. It is advised only to specify a device type, e.g., "iPhone Xʀ" and avoid explicit search by OS version.

      at SimulatorAllocDriver._queryDevices (node_modules/detox/src/devices/allocation/drivers/ios/SimulatorAllocDriver.js:177:13)
      at async SimulatorAllocDriver._groupDevicesByStatus (node_modules/detox/src/devices/allocation/drivers/ios/SimulatorAllocDriver.js:150:27)
      at async SimulatorAllocDriver._findOrCreateDevice (node_modules/detox/src/devices/allocation/drivers/ios/SimulatorAllocDriver.js:119:29)
      at async /Users/runner/work/hotline-bling-codealong/hotline-bling-codealong/node_modules/detox/src/devices/allocation/drivers/ios/SimulatorAllocDriver.js:43:14
      at async safeAsync (node_modules/detox/src/utils/safeAsync.js:4:13)
      at async /Users/runner/work/hotline-bling-codealong/hotline-bling-codealong/node_modules/detox/src/devices/allocation/DeviceRegistry.js:[6](https://github.com/upleveled/hotline-bling-codealong/actions/runs/7948582784/job/21698753711?pr=193#step:11:7)2:24
      at async ExclusiveLockfile.exclusively (node_modules/detox/src/utils/ExclusiveLockfile.js:41:15)
      at async SimulatorAllocDriver.allocate (node_modules/detox/src/devices/allocation/drivers/ios/SimulatorAllocDriver.js:42:18)
      at async /Users/runner/work/hotline-bling-codealong/hotline-bling-codealong/node_modules/detox/src/devices/allocation/DeviceAllocator.js:3[7](https://github.com/upleveled/hotline-bling-codealong/actions/runs/7948582784/job/21698753711?pr=193#step:11:8):[22](https://github.com/upleveled/hotline-bling-codealong/actions/runs/7948582784/job/21698753711?pr=193#step:11:23)

[eliottparis]

@karlhorky macos-13 runner does not have iPhone 13 simulators installed. Try running with iPhone 14 Pro instead.

[karlhorky]

Ohh thanks for the link, didn't know this information was documented, thanks!

I changed my simulator.device.type configuration to iPhone 14 Pro, and it worked 🙌 PR here if anyone's interested:

• https://github.com/upleveled/hotline-bling-codealong/pull/193

[eliottparis]

Hey @karlhorky, I noticed that your job is now passing with an iPhone 14 Pro simulator. What is currently struggling me is the time that takes detox to assign a test after it starts, It takes almost 8min in your case (10min for the project I working on), It was way shorter on macos-12, is this a regression?

14:16:17.451 detox[13467] B jest --config ./e2e/jest.config.cjs e2e
14:24:06.68 detox[13468] i appStarts.e2e.ts is assigned to 5B1B6B4F-C3E2-4D8B-A768-777D9C04EF15 (iPhone 14 Pro)

Does anyone know what can cause this?

[karlhorky]

I'm not sure what the cause of that would be (maybe indeed a regression in Detox?) but here are my numbers:

• detox test on macos-12: 2m 46s
• detox test on macos-13: 8m 24s

The command that is taking the longest seems to be this (but maybe not helpful, since this is probably doing a lot?):

jest --config ./e2e/jest.config.cjs e2e