swift-syntax icon indicating copy to clipboard operation
swift-syntax copied to clipboard

Build performance issue of swift-testing

Open Kyle-Ye opened this issue 1 year ago • 9 comments

Description

Before migrating to use "swift-testing", one of my package's average CI time is as the following

XCTest macOS 13 + Swift 5.9.2 Ubuntu + Swift 5.9.2
Swift Test(Debug+Covage) 45s 20s
Swift Test(Release) 34s 27s

Most the job will do its job less in 1 min.

However after fully migrate all my test case to swift-testing, the CI time exploded especially with Release configuration.

Expected behavior

It is expected the build time will increase due to the usage of swift macro. And 2~3 min looks normal and acceptable.

Increased to more than 10 min is definitely means something is wrong here IMO.

And my question is the following:

  1. Is this a known issue?
  2. Is this expected for Swift 5.9 (or 5.10) client only and the issue will not happen for Swift Snapshots toolchain?
  3. Is there a workaround for this currently?

Actual behavior

Testing macOS 13 + Swift 5.9.2 Ubuntu + Swift 5.9.2
Swift Test(Debug+Covage) 4min3s 3min24s
Swift Test(Release) 11min19s 12min37s

Steps to reproduce

No response

swift-testing version/commit hash

swift-testing 0.2

Swift & OS version (output of swift --version ; uname -a)

Swift 5.9.2

Kyle-Ye avatar Jan 01 '24 17:01 Kyle-Ye