kotest icon indicating copy to clipboard operation
kotest copied to clipboard

Test names containing line breaks are ignored when run individually

Open sndawson opened this issue 1 year ago • 1 comments

Which version of Kotest are you using 5.0.0 and greater.

I believe this is both a Kotest and a Kotest Intellij Plugin issue, so I have also created an issue there: https://github.com/kotest/kotest-intellij-plugin/issues/292

The problem appears to have been introduced in Kotest 5.0.0 from my testing.

I have seen the issue with Kotest versions 5.0.0, 5.5.5 and 5.81, Kotlin versions 1.6, 1.7 and 1.9 as well as JVM 11 and 17. The issue does not occur with Kotest version 4.6.4.

An example file:

import io.kotest.core.spec.style.BehaviorSpec

class MainTests : BehaviorSpec({
    Given("If you run from this Given, the test will run") {
        println("Given test 1")
        When(
            """
            But if you run from this When
            with line breaks,
            the test will be ignored
            """
        ) {
            println("When test 1")
            Then("It will also be ignored if you try to run it from here") {
                println("Then test 1")
            }
        }
    }
    Given("""A test case
        with a line break
        at a Given will also be ignored
        """
    ) {
        println("Given test 2")
        When("And same if you try to run it from here") {
            println("When test 2")
            Then("Same here") {
                println("Then test 2")
            }
        }
    }
    Given("""Test names with line breaks
        that trim any part of the string
        don't even get the run button
    """.trimMargin()) {
        println("Given test 3")
        When("And this test will be ignored even if you run it from here") {
            println("When test 3")
            Then("Same at this spot") {
                println("Then test 3")
            }
        }
    }
    Given("""Line
        breaks
        """
    ) {
        println("Given test 4")
        When("Shorter test name") {
            println("When test 4")
            Then("Shorter test name also") {
                println("Then test 4")
            }
        }
    }
})

Expected behaviour (as seen in 4.6.4) (the individual test runs): image

Behaviour with kotest version 5.X (all tests are ignored): image the output:

C:\Users\sdawson\.jdks\corretto-17.0.10\bin\java.exe -Didea.test.cyclic.buffer.size=1048576 "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2023.1.2\lib\idea_rt.jar=60675:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2023.1.2\bin" -Dfile.encoding=UTF-8 @C:\Users\sdawson\AppData\Local\Temp\idea_arg_file1953867495 io.kotest.engine.launcher.MainKt --spec MainTests --testpath "A test case
        with a line break
        at a Given will also be ignored
        " --reporter teamcity
Testing started at 7:30 PM ...

If you run from this Given, the test will run is excluded by test filter(s): Excluded by test path filter: 'A test case
        with a line break
        at a Given will also be ignored
        '

A test case with a line break at a Given will also be ignored is excluded by test filter(s): Excluded by test path filter: 'A test case
        with a line break
        at a Given will also be ignored
        '

Test names with line breaks that trim any part of the string don't even get the run button is excluded by test filter(s): Excluded by test path filter: 'A test case
        with a line break
        at a Given will also be ignored
        '

Line breaks is excluded by test filter(s): Excluded by test path filter: 'A test case
        with a line break
        at a Given will also be ignored
        '

Process finished with exit code 0

sndawson avatar Apr 05 '24 21:04 sndawson

A similar behavior can be observed if the test-name uses certain special characters, e.g. do some weird things with \$foobar

deveth0 avatar Aug 21 '24 08:08 deveth0