TestContext commands does not works under OneTimeTearDown function annotation

[CMALD]

    [OneTimeTearDown]
    public static void CleanupDriver()
    {
        TestContext.AddTestAttachment(@"C:\Screenshots\1.Jpeg" , "Description Test Attachment");
        TestContext.Out.WriteLine("Test End!!!");
        foreach(var key in Drivers.Keys)
        {
           Drivers[key].Quit();
        }
    }

When my function has the annotation: OneTimeTearDown, It seems that the TestContext commands does not works, And in the Test Output window, the attached file and the "Test End!!!" message does not displays.

Is it a bug or is this behavior expected under the OneTimeTearDown annotation?

e.g. I run my test by VS Test Adapter tool, (On "Test Explorer" window, right click on test name, and click on "Run Selected Test"), and I has NuGet Package : NUnit3TestAdapter v3.13.0

[CharliePoole]

Ah! That changes things a bit. I won't move this issue yet. You are running into a problem with two contributing causes...

1. You are adding the attachment in the context of the TestFixture, not an individual test case. OneTimeSetUp is about the fixture.

2. TestExplorer displays almost no info about suites or fixtures. It's all about test cases.

Doesn't your screenshot pertain to a single test case?

[CMALD]

I have 2 issuese:

1. TestContex output (text and attachments) does not works under [OneTimeTearDown] annotation
2. Description from TestContext.AddTestAttachment does not displays in Test Output window , In general, regardless of the annotation, (The attached link does appear, but as file name and not as description]

[CharliePoole]

Just to take care of the documentation part of this (since we are in the docs repo here) I updated the pages for OneTimeSetUp attribute, OneTimeTearDown attribute and TestContext so as to call attention to the problem of knowing what context you are in. @jnm2 @ChrisMaddock and others, could you please review the changes and correct or comment?

[CharliePoole]

@CMALD Regarding your first issue... TestContext output not working under OneTimeTearDown...

This is incorrect. It is working exactly as designed. Your code is adding the screen shots for every test to the context of the test fixture. I'm pretty sure this is working and you would find the information in the XML for the fixture.

However, if you want to see it under a runner which does not display info about the fixture then you have to add it to the indiviual test cases. Your code could, for example, be in a [TearDown] method, although it might require some modification there in order to post only the screen shots for hat test.

The NUnit VS adapter gives VS only the info that it will display, that is, info about test cases.

[CharliePoole]

@CMALD Regarding your seond issue... Description not displaying...

Is there some documentation that suggests the description will display in the output window? This would surprise me. I would expect the info to appear as part of the test case result itself. However, it's really a question for @OsirisTerje and I'm going ahead and transfering this to the adapter repository so he can deal with issue 2.

[CharliePoole]

@nunit/core-team I'm not crazy about how this beta GitHub transfer works. The original issue just disappears. OK in some cases, but not in others.

[CMALD]

@CharliePoole Regarding my second issue - Description not displaying: I did not understand where I was supposed to see the description?

[CharliePoole]

Me neither! That's why this had to be transferred to the adapter repo where @OsirisTerje will see it and hopefully tell us! Personally, as a user, I think the Output window is a bad place to display non-exceptional info about individual tests.

I haven't worked on the adapter for a few years, but my memory (perhaps faulty) was that we tell Microsoft the description and they display it somewhere. @OsirisTerje ?

@CMALD Just to be clear... I don't believe you will ever see a description that applies to a suite or fixture. As previously stated, the TestExplorer IDE deals with individual test cases. There's nothing that NUnit or the adapter can do about that.

Do you see a description that you add to an individual test case?

[CMALD]

What the meaning of: "description that you add to an individual test case?" give me an example

[CMALD]

I do not think it's such an illogical display in the output window description for the attachment. Like any print to the output window, it is designed to make it easier for the developer to debug the code That's how I use it anyway ...

[CharliePoole]

As an example...

    [TearDown]
    public static void CleanupDriver()
    {
        TestContext.AddTestAttachment(@"C:\Screenshots\1.Jpeg" , "Description Test Attachment");
        TestContext.Out.WriteLine("Test End!!!");
        foreach(var key in Drivers.Keys)
        {
           Drivers[key].Quit();
        }
    }

This, because it's TearDown rather than OneTimeTearDown, creates a description in the context of the individual test case rather than the test representing the whole fixture. I kept all the same code as your example but the handling of the driver would obviously have to change. I don't know enough about the rest of your code to say exactly how it would change.

[CharliePoole]

Many users add descriptions to all or the majority of their tests. Displaying those descriptions in the Output window would obviously be confusing.

However, I see you are not talking about those descriptions but the descriptions associated with attachments. That's a much more limited case. We do frequently use the output window for things that can't be displayed in any other way, so we should wait for @OsirisTerje, who knows how this currently works, to respond.

[OsirisTerje]

@CMALD The attachment descriptions are added to the test results, and sent forward to the test host. But currently I can't see it is doing anything meaningful with these descriptions (meaning, I can't see it displayed anywhere, neither for passed nor failed tests).
I'll ping the PG about this. I also noted that MSTest doesn't seem to have any straight forward way of adding descriptions to an attachment, even if the underlying objects have that facility (which is what we're using).

As @CharliePoole says, adding this information to the test output is not what we normally do. We could add a featureflag to enable such an output, but it would anyway need to be set through a runsettings file. Having this on by default would give extra output I suspect many would not want.

[TJEvans]

Does this prevent Test attachments from being added to the results file at the path /test-suite/attachments/attachment as supported here? I have not found a way to accomplish this yet.

This is of interest due to Microsoft's documentation about adding attachments to their Test Results

[OsirisTerje]

@TJEvans You're pointing to the NUnit Test results, but the adapter reports to the VSTest system and the Microsoft test result file.
Can you elaborate on what you try to achieve and how your system is set up?

[TJEvans]

Im looking for how test attachments are supported at the test suite level. We have custom logger that is generating an html report of the test run, and these files need to be called out on the /TestRun/ResultSummary/ResultFiles/ResultFile.Attributes["path"].Value path in the TRX.

I am able to get the test report uploaded by declaring it a test attachment, TestContext.AddTestAttachment(f.FullName), for each test, in the TearDown, but this not ideal. I see doing this in the OneTimeTearDown is not having the expected result as described above. No ResultsFiles are listed in the ResultSummary node.

[tomwaitforitmy]

This is exactly what I need. I am looking for a possibility to attach a file to test suite/fixture instead of a single test. In our case, we use NUnit to perform system UI tests (with LeanFT). In system tests, the whole "every test runs independently" becomes impossible and changes some stuff. In our case, the UI testing framework generates a report for the complete run. This report does not always belong to a single but, instead belongs to a test suite (e.g. "StartApplication", "EditFile", "EndApplication"). VSTS offers the possibility to show attachments of a test suite. So why not offer "AddTestFixtureAttachment()"? Let me know if you need more input.