lucenenet icon indicating copy to clipboard operation
lucenenet copied to clipboard

[Slow] attribute runs the code in BeforeClass() and AfterClass() even when the `tests:slow` property is set to `false`

Open NightOwl888 opened this issue 1 year ago • 0 comments

When the SlowAttribute in Lucene.Net.TestFramework is placed on a LuceneTestCase-derived class, it runs the code in the BeforeClass() and AfterClass() methods (which are decorated with [OneTimeSetUp] and [OneTimeTearDown]) even when the system property tests:slow is set to false. This causes these disabled tests to run code, taking up extra testing time. Since the purpose of these settings is to optimize the test run, this undesired behavior is a bug.

I suspect (but haven't confirmed) that the following other attributes also have a similar issue, since they all follow the same basic design.

  • AwaitsFixAttribute
  • NightlyAttribute
  • WeeklyAttribute

Ideally, these would completely suppress any actions from occurring when applied at the class level when their associated system property is disabled. System properties are statically loaded by the test framework in LuceneTestCase.

Note: There is some better info here on how to specify system properties and command line example here. No general docs for this yet.

These attributes use NUnit's extensibility through attribute interfaces IApplyToTest, IApplyToContext, and IWrapTestMethod to function. NUnit scans for attributes that implement these interfaces in order to execute them.

Ideally, we find a way to suppress the [OneTimeSetUp] and [OneTimeTearDown] calls somehow. But these execution calls are owned by NUnit.

Failing that we may be able implement our own attributes with the same names that implement the same interfaces. When these attributes are placed as nested classes of LuceneTestCase, they effectively "override" the attributes in the NUnit.Framework namespace for LuceneTestCase or any class that inherits it.

Note: We use this approach already for our custom [TestFixtureAttribute]. There may be some way to use that custom attribute to fix this where we have direct control over the code. It looks like we are already intercepting these calls here, but I am not sure whether that gives us ownership over all of them.

There is some custom attribute documentation, but it doesn't actually include IWrapTestMethod or other interfaces that are available in the source code.

NightOwl888 avatar Nov 03 '22 10:11 NightOwl888