snapshooter
snapshooter copied to clipboard
Running snapshot test in release mode fails
Describe the bug When running a snapshot test in the release configuration it fails with the following error:
Test.Tests.SnapshooterTest.MultiplySnapshotTest
Snapshooter.Exceptions.SnapshotTestException : The snapshot full name could not be evaluated. This error can occur, if you use the snapshot match within a async test helper child method. To solve this issue, use the Snapshot.FullName directly in the unit test to get the snapshot name, then reach this name to your Snapshot.Match method.
at Snapshooter.Xunit.XunitSnapshotFullNameReader.ReadSnapshotFullName() in /home/vsts/work/1/s/src/Snapshooter.Xunit/XunitSnapshotFullNameReader.cs:line 54
at Snapshooter.SnapshotFullNameResolver.ResolveSnapshotFullName(String snapshotName, String snapshotNameExtension) in /home/vsts/work/1/s/src/Snapshooter/SnapshotFullNameResolver.cs:line 69
at Snapshooter.Xunit.Snapshot.Match(Object currentResult, Func`2 matchOptions) in /home/vsts/work/1/s/src/Snapshooter.Xunit/Snapshot.cs:line 145
To Reproduce
- Create this class and it's corresponding test:
public class Multiplier
{
public float MultiplyByThree(float input)
{
return input * 3;
}
}
public class SnapshooterTest
{
[Fact]
public void MultiplySnapshotTest()
{
var multiplier = new Multiplier();
float result = multiplier.MultiplyByThree(5);
Snapshot.Match(result);
}
}
- Run the test (debug configuration)
- Verify your snapshot
- Run the test again (release configuration)
Expected behavior The test should pass both with the debug and the release configuration.
Actual behavior The test only passed in debug mode.
Environment:
- OS: macOs Mojave
- Package Versions:
- Snapshooter: 0.4.5
- xUnit: 2.4.1
To resolve the location of the Testfile we have to use the StackFrames which are only available in Debug mode. When in future a TextContext should be available in xUnit we could fix this. @maracuja-juice Can you explain what use case do you have to execute the tests in Release mode?
We run the build in our CI in release mode and test against that. Because we want to test the code the way it will actually run as, which is in Release configuration.
In your linked issue they linked to this doc page: https://msdn.microsoft.com/en-us/library/system.reflection.assembly.codebase%28v=vs.110%29.aspx Is there a problem with that in Release mode or are there other limitations that won't work for the use case of this library?
Is there a workaround for this?