kotest icon indicating copy to clipboard operation
kotest copied to clipboard

Nested Test Names broken in in Kotest 5 (worked in Kotest 4)

Open hosswald opened this issue 2 years ago • 9 comments

Which version of Kotest are you using 5.3.1

Nested test names are broken in Kotest 5+:

class `Level 0` : FreeSpec({
    "Level 1" - {
        "Level 2" - {
            "Level 3.1" {
            }
            "Level 3.2" {
                fail("")
            }
        }
    }
})

image

In Gradle it looks like this:
> Task :tests:test

Level 0 > Level 0 > Level 0 > Level 3.2 FAILED
    java.lang.AssertionError at KotestBroken.kt:10
        Caused by: java.lang.AssertionError at KotestBroken.kt:10

It does not matter which Testing Style (that supports nesting) I use. I would expect the test report to reflect the hierarchy, as it did in Kotest 4.6.3 (only "failed" (yellow) vs "error" (red) could have been more consistent): image

hosswald avatar Jun 21 '22 10:06 hosswald

#2730 seems related

hosswald avatar Jun 21 '22 10:06 hosswald

Test reports exported fron IntelliJ IDEA 2022.1.2 UE with Kotest Plugin 1.2.58-IC-2022.1 TestResult_Kotest_4_6_3.xml.txt TestResult_Kotest_5_3_1.xml.txt

hosswald avatar Jun 21 '22 10:06 hosswald

I can't duplicate with gradle 7.4.2. Which version of gradle are you using ?

> Task :template-server:test

<...removed stack traces...>

com.sksamuel.template.server.Level 0 > Level 1 > Level 2 > com.sksamuel.template.server.Level 0.Level 3.2 FAILED

sksamuel avatar Jul 04 '22 00:07 sksamuel

And this is 5.3.2

sksamuel avatar Jul 04 '22 00:07 sksamuel

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Aug 12 '22 00:08 stale[bot]

@hosswald I cannot duplicate this. Is it still happening ?

sksamuel avatar Aug 28 '22 20:08 sksamuel

@sksamuel sorry, I missed your mention in July. Please try with Gradle version 6.6.1. I just tested, with Gradle 7.5.1 I don't have the bug (but gradle major version updates are not always an option).

hosswald avatar Aug 29 '22 09:08 hosswald

  • Gradle 6.8.3 (latest major 6 version): broken
  • Gradle 7.0: not broken

hosswald avatar Aug 29 '22 09:08 hosswald

We're unlikely to add a workaround for a bug in gradle 6.x due to the age of that release and the number of junit bugs it contains.

sksamuel avatar Aug 29 '22 09:08 sksamuel

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Oct 01 '22 05:10 stale[bot]

Hello,

I have simmiliar problem with kotest 5.4.2. Running via Idea

class MyTests : FunSpec({
    context("this outer block is enabled") {
        test(" name") {
            // test here
        }
    }
})

does not prepend this outer block is enabled

to me problem seems to be in this function.

   /**
    * Adds a test case to this context.
    */
   suspend fun test(name: String, test: suspend TestScope.() -> Unit) {
      registerTest(TestName(name), false, null, test)
   }

which does not append name from FunSpecContainerScope.

Should be imho changed into

   /**
    * Adds a test case to this context.
    */
   suspend fun test(name: String, test: suspend TestScope.() -> Unit) {
      rregisterTest(TestName(testScope.testCase.descriptor.name.testName, name), false, null, test)
   }

stengvac avatar Nov 03 '22 21:11 stengvac

@stengvac please test with latest version of kotest and raise a new issue if the problem persists, since this turned out to relate to a gradle 6.x bug. Preferably along with a screenshot showing the problem.

Kantis avatar Nov 03 '22 21:11 Kantis