snapshooter icon indicating copy to clipboard operation
snapshooter copied to clipboard

Running snapshot test in release mode fails

Open panmona opened this issue 5 years ago • 3 comments

Describe the bug When running a snapshot test in the release configuration it fails with the following error:

Test.Tests.SnapshooterTest.MultiplySnapshotTest

Snapshooter.Exceptions.SnapshotTestException : The snapshot full name could not be evaluated. This error can occur, if you use the snapshot match within a async test helper child method. To solve this issue, use the Snapshot.FullName directly in the unit test to get the snapshot name, then reach this name to your Snapshot.Match method.
   at Snapshooter.Xunit.XunitSnapshotFullNameReader.ReadSnapshotFullName() in /home/vsts/work/1/s/src/Snapshooter.Xunit/XunitSnapshotFullNameReader.cs:line 54
   at Snapshooter.SnapshotFullNameResolver.ResolveSnapshotFullName(String snapshotName, String snapshotNameExtension) in /home/vsts/work/1/s/src/Snapshooter/SnapshotFullNameResolver.cs:line 69
   at Snapshooter.Xunit.Snapshot.Match(Object currentResult, Func`2 matchOptions) in /home/vsts/work/1/s/src/Snapshooter.Xunit/Snapshot.cs:line 145

To Reproduce

  1. Create this class and it's corresponding test:
public class Multiplier
{
    public float MultiplyByThree(float input)
    {
        return input * 3;
    }
}
public class SnapshooterTest
{
    [Fact]
    public void MultiplySnapshotTest()
    {
        var multiplier = new Multiplier();
        float result = multiplier.MultiplyByThree(5);
        
        Snapshot.Match(result);
    }
}
  1. Run the test (debug configuration)
  2. Verify your snapshot
  3. Run the test again (release configuration)

Expected behavior The test should pass both with the debug and the release configuration.

Actual behavior The test only passed in debug mode.

Environment:

  • OS: macOs Mojave
  • Package Versions:
    • Snapshooter: 0.4.5
    • xUnit: 2.4.1

panmona avatar Apr 26 '19 09:04 panmona

To resolve the location of the Testfile we have to use the StackFrames which are only available in Debug mode. When in future a TextContext should be available in xUnit we could fix this. @maracuja-juice Can you explain what use case do you have to execute the tests in Release mode?

philbir avatar May 29 '19 10:05 philbir

We run the build in our CI in release mode and test against that. Because we want to test the code the way it will actually run as, which is in Release configuration.

In your linked issue they linked to this doc page: https://msdn.microsoft.com/en-us/library/system.reflection.assembly.codebase%28v=vs.110%29.aspx Is there a problem with that in Release mode or are there other limitations that won't work for the use case of this library?

panmona avatar May 29 '19 20:05 panmona

Is there a workaround for this?

papadi avatar Oct 06 '22 07:10 papadi