swift-snapshot-testing icon indicating copy to clipboard operation
swift-snapshot-testing copied to clipboard

Running test repeatedly results in duplicate snapshot being created.

Open Deco354 opened this issue 3 years ago • 2 comments

Describe the bug Running a snapshot test repeatedly with xcode's new 'run test repeatedly' functionality results in additional snapshots being created for that test.

We've a wrapper around this library which I've posted below although the identifier we're passing in to the testName does not include the iteration number at the real testSnapshot callsite.

    public func testSnapshot(_ view: UIView,
                             identifier: String = #function,
                             snapshotType: SnapshotType = .image,
                             precision: CGFloat = 0.95,
                             file: StaticString = #file,
                             line: UInt = #line,
                             waiting: TimeInterval = 0,
                             isArchitectureDependent: Bool = false) {
        semaphore.wait()
        var testID = identifier
        if isArchitectureDependent && (Architecture.isArm64 || Architecture.isRosettaEmulated) {
            testID.append("-arm64")
        }
        
        switch snapshotType {
        case .image:
            assertSnapshot(matching: view,
                           as: .wait(for: waiting, on: .image(precision: Float(precision))),
                           record: isRecording,
                           file: file,
                           testName: testID,
                           line: line)
            
        case .recursiveDescription:
            assertSnapshot(matching: view,
                           as: .wait(for: waiting, on: .recursiveDescription()),
                           record: isRecording,
                           file: file,
                           testName: testID,
                           line: line)
        }
        semaphore.signal()
    }

To Reproduce Take any snapshot test right click on it and run "test repeatedly". At a certain iteration a number matching this iteration (2 in the screenshots below) will be appended to the snapshot reference image name causing an additional snapshot to be saved and the tests to fail. image

image

Expected behavior I'd expect the test to use only one snapshot no matter how many time they're run.

Environment

  • swift-snapshot-testing version [e.g. 1.8.1]
  • Xcode 13.2
  • Swift 5.5
  • OS: iOS 15

Deco354 avatar Feb 22 '22 14:02 Deco354

A workaround is to set the named parameter

simondelphia avatar Jul 05 '22 22:07 simondelphia

@simondelphia @Deco354 Thank you. You guys really saved my day. Running TC on Xcode Cloud will hit the issue.

ghost avatar Jul 07 '22 07:07 ghost

@stephencelis Can we reopen this? Using the named parameter is a work-around rather than a fix. It won't be immediately obvious to anyone other than those within this thread.

Deco354 avatar Sep 22 '22 09:09 Deco354

@Deco354 this was closed by merging #585. Are you still having issues running repeatedly in yesterday's 1.10.0 release?

stephencelis avatar Sep 22 '22 16:09 stephencelis

Sorry I must have skimmed this thread and missed #585 completely. Thanks for getting this merged and thanks @krzysztofpawski for taking the time to fix this 🍾

Deco354 avatar Sep 23 '22 09:09 Deco354