File Save Permissions Error

[Sherlouk]

Hi 👋

First off just wanted to say thanks for the project, it's been a breeze to setup and super refreshing! -- Good work!

My question is currently whether or not this library is suppose to support physical devices or if the intended use is limited to Simulators?

When running on a physical device I get the below error when fileManager.createDirectory is called. The same is not true when ran from a simulator (which works as expected, without errors).

Error Domain=NSCocoaErrorDomain Code=513 "You don’t have permission to save the file “CaseNameTests” in the folder “__Snapshots__”." UserInfo={NSFilePath=/Users/name/path/to/repo/ModuleTests/__Snapshots__/CaseNameTests, NSUnderlyingError=0x1c28585d0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}

This appears to be the same issue referenced here

Thanks in advance 😄

[stephencelis]

Hi @Sherlouk! We recently added snapshotDirectory to the base verifySnapshotHelper: https://github.com/pointfreeco/swift-snapshot-testing/blob/5ae8db2a9cad57a698883fdad9e40b72654fd3dc/Sources/SnapshotTesting/AssertSnapshot.swift#L161

This means you can write your own assertSnapshot helper to change the directory in which the images are saved:

https://github.com/pointfreeco/swift-snapshot-testing/blob/5ae8db2a9cad57a698883fdad9e40b72654fd3dc/Sources/SnapshotTesting/AssertSnapshot.swift#L117-L143

We know this is a bit hidden and that the ergonomics aren't great, so we hope to improve the experience in the future.

[Sherlouk]

Apologies in the delay getting back to you @stephencelis - I've updated my references to assert with my own implementation loosely following the example you showed above using a different directory in the top-level of my projects file structure but unfortunately am getting the same error (just paths updated to a different location).

Again, this is only happening when ran from a device - working fine from Simulator 😞Is there something different happening in file permissions when running from device?

[Sherlouk]

I'll add that I've just done a further investigation with uber/ios-snapshot-test-case and they're getting the exact same issue where on device it doesn't save images, and on simulator it does.

From looking at the code it appears both implementations do more or less the same thing (using FileManager to create directories, and then image.save(to:) into the file location.

Interested in your opinion on this?

[stephencelis]

Hi @Sherlouk, sorry for the delay! While we've written a ton of snapshot tests we've never had the need to do so outside of a simulator. Could you describe your use case a bit more? We'd be happy to accommodate this at the library level if it's possible and useful, but we haven't had the need to implement such functionality ourselves.

[Sherlouk]

No problem! More than happy to try and explain our use case a bit more,

Internally we have a CI platform that supports a large number of physical devices ranging from phones through tablets of varying sizes and configurations. We take advantage of Hive CI to run our integration tests on and were hoping to also take advantage of it for our snapshot tests too.

In general this comes down to a bit of reassurance from our testing department as we have seen various little differences in the past when it comes to Simulator vs Physical Device and was hoping to do the comparison on the closest thing to real world as possible

Hope that makes sense?

[mathieutozer]

I'm running into this error while trying to see snapshots to disk in a macOS test environment. Is there an extra set of config for Mac testing?

[Mark-s99]

Same problem here. I just created a new macOS app project, added the Snapshots project and tried to run a simple test.

I think the problem is that the test run sandboxed on macOS. The question is how to extend the app sandbox so that the unit tests can write the snapshots to a given directory (and not only the temporary directory).

To reproduce:

• Open Xcode, create a new Cocoa application
• Add the Snapshots project
• Create a simple test : assertSnapshot(matching: "Hello", as: .lines)

Result: Test fails with: "failed - You don’t have permission to save the file “BasicTests” in the folder “Snapshots”"

[rsaarloos]

Hi,

Also running into the same issue, only when running the Snapshot test on a real device. We needed to switch to testing on a real device as we incorporated a SDK that does not support Simulators (no x86_64 slice available)

Any ideas on how to resolve this

[darknoon]

I was able to work around this on macOS by disabling app sandboxing for the test host app.

in Entitlements.plist:

+        <key>com.apple.security.app-sandbox</key>
+        <false/>

As far as I see it, if you're running on device, you need to run some kind of server or LLDB plugin on the mac to accept the screenshots and write them back to the filesystem (since writing them on device would obviously be no use).

[BenoitCaron]

Hi @stephencelis,

I tried writing my own assertSnapshot in order to change the output directory, and my error is similar, but not exactly the same: failed - You can’t save the file “MyTests” because the volume is read only.

I tried replicating the default behaviour, but the error still occurs.

public func snapshotDirectory(file: StaticString) -> String {
    let fileUrl = URL(fileURLWithPath: "\(file)", isDirectory: false)
    let fileName = fileUrl.deletingPathExtension().lastPathComponent

    return fileUrl.deletingLastPathComponent()
        .appendingPathComponent("__Snapshots__")
        .appendingPathComponent(fileName)
        .absoluteString
 }

public func myAssertSnapshot<Value, Format>(
    matching value: @autoclosure () throws -> Value,
    as snapshotting: Snapshotting<Value, Format>,
    named name: String? = nil,
    record recording: Bool = false,
    timeout: TimeInterval = 5,
    file: StaticString = #file,
    testName: String = #function,
    line: UInt = #line
) {
    let failure = try verifySnapshot(
        matching: value(),
        as: snapshotting,
        named: name,
        record: recording,
        snapshotDirectory: snapshotDirectory(file: file),
        timeout: timeout,
        file: file,
        testName: testName
    )
    guard let message = failure else { return }
    XCTFail(message, file: file, line: line)
}

This is the default code, in AssertSnapshot.swift:

      let fileUrl = URL(fileURLWithPath: "\(file)", isDirectory: false)
      let fileName = fileUrl.deletingPathExtension().lastPathComponent

      let snapshotDirectoryUrl = snapshotDirectory.map { URL(fileURLWithPath: $0, isDirectory: true) }
        ?? fileUrl
          .deletingLastPathComponent()
          .appendingPathComponent("__Snapshots__")
          .appendingPathComponent(fileName)

It seems snapshotDirectory.map { URL(fileURLWithPath: $0, isDirectory: true) } and fileUrl.deletingLastPathComponent().appendingPathComponent("__Snapshots__").appendingPathComponent(fileName) don't output the exact same thing:

• snapshotDirectory.map { URL(fileURLWithPath: $0, isDirectory: true) } outputs file:/Users/benoit/myApp/myApp-iOSTests/__Snapshots__/MyTests/ -- file:///
  • this one doesn't work
• fileUrl.deletingLastPathComponent().appendingPathComponent("__Snapshots__").appendingPathComponent(fileName) outputs file:///Users/benoit/myApp/myApp-iOSTests/__Snapshots__/MyTests/
  • this one works

Why is there a difference? Am I missing something?

Edit: I forked the library and replaced the String? snapshotDirectory parameter with a URL? snapshotDirectory parameter, and it does work as intended. Should I create a PR?

[fabstu]

I also have this issue with MacOS. @BenoitCaron does this solve it for MacOS too?

[BenoitCaron]

Sorry I wouldn't know, my project is iOS.

[bzhoek]

Hi @BenoitCaron and @fabstu , I ran into the same issue on macOS. What solved it for me was to convert the URL to string with .path() instead of ..absoluteString() in my custom myAssertSnapshot, like so

let snapshotDirectory = createSnapshotDirectory(file: file) // returns URL

let failure = verifySnapshot(
      matching: value,
      as: snapshotting,
      named: name,
      record: recording,
      snapshotDirectory: snapshotDirectory.path,
      timeout: timeout,
      file: file,
      testName: testName,
      line: line
    )

[fredybp]

I fixed this by selecting Expand Variables Based On and selecting the based Target. It fixed for me.

[jeff77731]

Same problem

Doesn't have permission to create image in Snapshots directory

Any solutions?

[BenoitCaron]

That happens to me from time to time, when changing branches. Deleting the whole directory and recreating snapshots usually works for me.

[parikhparth23]

I was also getting the same error when using https://github.com/cashapp/AccessibilitySnapshot. For me the issue was absoluteString fixtureProvider.snapshotDirectory(forTestFile: file).absoluteString

changing to fixtureProvider.snapshotDirectory(forTestFile: file).path

worked for me.

this forum helped me - https://www.hackingwithswift.com/forums/ios/trying-to-create-directory-in-documents-folder-fails/15774/15788

[TomaszLizer]

I am having same issue as others described for iOS device. I wanted to check real device, because for some reason code coverage is not gathered on simulator. It is gathered just fine for physical device, but I am getting failed - You don’t have permission to save the file “CalendarLargeWidgetViewSnapshotTests” in the folder “__Snapshots__”. failure in tests :)