Feature request: Cascading snapshot storage

I'm looking into integrating SnapshotTesting for SourceKitten's test fixtures and I'm trying to replicate the current behaviour of the snapshotting strategy I'm replacing.

First some context: SourceKitten interacts to SourceKit and libclang versions that ship (and change!) across Swift versions and across supported platforms. So the snapshots are dependent on the swift version and OS.

The simplest way I've found to integrate SnapshotTesting with this "test matrix" is to specify these variants in the name:

enum SnapshotTesting {
#if os(Linux)
    static let swiftVersionAndOSName = "\(SwiftVersion.current.rawValue)-Linux"
#else
    static let swiftVersionAndOSName = "\(SwiftVersion.current.rawValue)"
#endif
}

func assertSourceKittenSnapshot<Value, Format>(
  matching value: @autoclosure () throws -> Value,
  as snapshotting: Snapshotting<Value, Format>,
  named name: String? = SnapshotTesting.swiftVersionAndOSName,
  record recording: Bool = false,
  timeout: TimeInterval = 5,
  file: StaticString = #file,
  testName: String = #function,
  line: UInt = #line
) {
    assertSnapshot(
        matching: try value(),
        as: snapshotting,
        named: name,
        record: recording,
        timeout: timeout,
        file: file,
        testName: testName,
        line: line
    )
}

This creates a separate snapshot for each Swift version + OS combination (only macOS + Linux are currently supported).

However, this makes it hard to compare the output and identify the differences across Swift versions and/or OS combinations.

This is also somewhat wasteful because often times the snapshots are identical across some subset of Swift versions & OS combinations, but in those cases a new duplicate file is still created with the exact contents of others.

That's why currently, without using SnapshotTesting, I'm using a strategy where my fixture comparison function progressively walks through the possible combinations and if it finds a match, it will compare the output to that matched file. In my equivalent of a recording mode, if the comparison fails, then it will create a new snapshot file with the current Swift version & OS.

I can probably cobble up a similar scheme on top of SnapshotTesting in my codebase, but I think that this type of progressive or cascading storage strategy would be broadly useful in a lot of similar but sometimes slightly different situations. For example:

• Rendered images of HTML content across different web browsers
• Localized content across similar locales (en_CA & en_US)
• Optimizable content across various levels of optimization (e.g. a code compiler)
• The same input passed through a matrix of different providers (e.g. weather data)

I was curious if you've considered something like this before and if you had any design ideas for how to integrate this type of feature. Or even if you think this is a reasonable feature for SnapshotTesting to support natively vs recommending that users configure it this way in their own projects.

@stephencelis @mbrandonw I'm curious what your thoughts are on something like this. Do you have any use cases that fit this pattern of a matrix of many almost-always-the-same fixtures in your projects that would benefit from this?

We think there's definitely use for this, sorry for the delay in chatting! Similarly in iOS view-based snapshots, people have requested easier support based off the simulator, OS snapshot in. We're in the process of triaging a lot of these issues, so I'll be grouping this one with those.