Swift package unit test failing on hosted macOS image but passing locally

[roanutil]

Description

A swift package CoreDataRepository on branch non-main-queue is failing two specific unit tests on the hosted image but passing on my local computer. Hosted environments macos-11 Xcode 12.4, 12.5.1, and 13.2; macos-12 Xcode 13.4.

Local environment is M1 Pro SoC, macOS 12.5, Xcode 13.4.1, 13.4, 13.2.0. Additionally, tests were run with Xcode opened through Rosetta and against 'My Mac (Rosetta)'.

Platforms affected

• [ ] Azure DevOps
• [X] GitHub Actions

Virtual environments affected

• [ ] Ubuntu 18.04
• [ ] Ubuntu 20.04
• [ ] Ubuntu 22.04
• [ ] macOS 10.15
• [X] macOS 11
• [X] macOS 12
• [ ] Windows Server 2019
• [ ] Windows Server 2022

Image version and build link

20220815.1 - https://github.com/roanutil/CoreDataRepository/actions/runs/2872537441

20220809.1 - https://github.com/roanutil/CoreDataRepository/runs/7871453504 (Xcode 13.4 only)

Is it regression?

No

Expected behavior

All tests pass like on local computer

Actual behavior

The same two tests fail on specific images and Xcode combinations.

Repro steps

Run the 'CI' workflow in the CoreDataRepository\non-main-queue, CoreDataRepository\non-main-queue-macos-12-runner repository/branches.

[al-cheb]

Hello @roanutil. We will take a look at it.

[dmitry-shibanov]

Hello @roanutil. Sorry for the late response. I could resolve some tests falling by increasing timeout for the requests tests. Could you please try to increase timeouts for the requests tests or mock requests ?

[roanutil]

@dmitry-shibanov I tried increasing all timeouts to 60 seconds but still get some failures. Each test case should take very little time. At least on my local MBP, it's basically instantaneous.

Test Suite 'All tests' started at 2022-10-09 23:42:16.719 Test Suite 'CoreDataRepositoryPackageTests.xctest' started at 2022-10-09 23:42:16.719 Test Suite 'AggregateRepositoryTests' started at 2022-10-09 23:42:16.719 Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testAverageSuccess]' started. Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testAverageSuccess]' passed (0.004 seconds). Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testCountSuccess]' started. Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testCountSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testMaxSuccess]' started. Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testMaxSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testMinSuccess]' started. Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testMinSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testSumSuccess]' started. Test Case '-[CoreDataRepositoryTests.AggregateRepositoryTests testSumSuccess]' passed (0.001 seconds). Test Suite 'AggregateRepositoryTests' passed at 2022-10-09 23:42:16.729. Executed 5 tests, with 0 failures (0 unexpected) in 0.009 (0.010) seconds Test Suite 'BatchRepositoryTests' started at 2022-10-09 23:42:16.729 Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testAltDeleteSuccess]' started. Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testAltDeleteSuccess]' passed (0.002 seconds). Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testAltUpdateSuccess]' started. Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testAltUpdateSuccess]' passed (0.002 seconds). Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testCreateSuccess]' started. Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testCreateSuccess]' passed (0.002 seconds). Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testDeleteSuccess]' started. Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testDeleteSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testInsertFailure]' started. Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testInsertFailure]' passed (0.004 seconds). Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testInsertSuccess]' started. Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testInsertSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testReadSuccess]' started. Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testReadSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testUpdateSuccess]' started. Test Case '-[CoreDataRepositoryTests.BatchRepositoryTests testUpdateSuccess]' passed (0.001 seconds). Test Suite 'BatchRepositoryTests' passed at 2022-10-09 23:42:16.744. Executed 8 tests, with 0 failures (0 unexpected) in 0.015 (0.015) seconds Test Suite 'CRUDRepositoryTests' started at 2022-10-09 23:42:16.744 Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testCreateSuccess]' started. Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testCreateSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testDeleteFailure]' started. Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testDeleteFailure]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testDeleteSuccess]' started. Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testDeleteSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testReadFailure]' started. Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testReadFailure]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testReadSubscriptionSuccess]' started. Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testReadSubscriptionSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testReadSuccess]' started. Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testReadSuccess]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testUpdateFailure]' started. Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testUpdateFailure]' passed (0.001 seconds). Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testUpdateSuccess]' started. Test Case '-[CoreDataRepositoryTests.CRUDRepositoryTests testUpdateSuccess]' passed (0.001 seconds). Test Suite 'CRUDRepositoryTests' passed at 2022-10-09 23:42:16.754. Executed 8 tests, with 0 failures (0 unexpected) in 0.009 (0.009) seconds Test Suite 'FetchRepositoryTests' started at 2022-10-09 23:42:16.754 Test Case '-[CoreDataRepositoryTests.FetchRepositoryTests testFetchSubscriptionSuccess]' started. Test Case '-[CoreDataRepositoryTests.FetchRepositoryTests testFetchSubscriptionSuccess]' passed (0.002 seconds). Test Case '-[CoreDataRepositoryTests.FetchRepositoryTests testFetchSuccess]' started. Test Case '-[CoreDataRepositoryTests.FetchRepositoryTests testFetchSuccess]' passed (0.001 seconds). Test Suite 'FetchRepositoryTests' passed at 2022-10-09 23:42:16.756. Executed 2 tests, with 0 failures (0 unexpected) in 0.003 (0.003) seconds Test Suite 'CoreDataRepositoryPackageTests.xctest' passed at 2022-10-09 23:42:16.757. Executed 23 tests, with 0 failures (0 unexpected) in 0.036 (0.037) seconds Test Suite 'All tests' passed at 2022-10-09 23:42:16.757. Executed 23 tests, with 0 failures (0 unexpected) in 0.036 (0.038) seconds

[mikhailkoliada]

@reupen Hello! As there has been no activity for a long time and multiple image versions have been released since, I am going to close the issue. If you still can reproduce the problem feel free to reach us again and reopen the ticket

[reupen]

I have not had any involvement in this. Perhaps you meant @roanutil ?