kotest

kotest copied to clipboard

Thread to contain some ideas for what we should change / add to 6.0

Breaking

• ~Disable classpath scanning by default for extensions - this causes slowdown at startup and can lead [to bugs] (https://github.com/kotest/kotest/issues/3852) with classgraph/reflection etc.~
• Update the shouldBe logic to use a fixed set of collection types, instead of anything that implements Iterable.
• ~Investigate if we can drop TestType.Dynamic~
• ~Remove all deprecated functions~
• Remove the old labels support from property testing in favor of Classifiers
• Change Printed output type to better support copy n pasting
• Unify before/after listeners with property tests including on shrink runs.
• ~Revisit the coroutine heirarchy for all tests~
• ~Consider removing or changing the way finalizeSpec works.~

Not Breaking

• ~Yaml matchers to mirror Json Matchers~
• ~Complete our concurrency support and remove experimental flags~
• Allow re-run of shrunken property tests directly
• ~Consider looking inside the package of the test for a well known config class name to avoid classpath scanning, to support config per package, and to avoid needing to use a well known config class package name.~
• ~Update the kotest plugin to run individual tests in native and js.~
• ~Use Kotlin 2.0~
• Expand the use of the Equality typeclass to all comparator style assertions and merge Eq and Equality into a shared typeclass.
• ~Consider making use of kotlin-power-assert which will become part of the standard compiler plugin set (KT-63607)~
• ~New before/after callback that runs at the nested container level~
• ~https://github.com/kotest/kotest/issues/4460~

• Consider making use of kotlin-power-assert which will become part of the standard compiler plugin set (KT-63607).

• Consider making use of kotlin-power-assert which will become part of the standard compiler plugin set (KT-63607).

Should we ask them to consider supporting other types of function names, so we could make it work with shouldBe ?

Seems like functionality like this is already there: https://github.com/bnorm/kotlin-power-assert

The Gradle plugin allows configuring the functions which should be transformed with a list of fully-qualified function names.

// Kotlin DSL
configure<com.bnorm.power.PowerAssertGradleExtension> {
  functions = listOf("kotlin.assert", "kotlin.test.assertTrue")
}

Seems like functionality like this is already there: https://github.com/bnorm/kotlin-power-assert

The Gradle plugin allows configuring the functions which should be transformed with a list of fully-qualified function names.

// Kotlin DSL
configure<com.bnorm.power.PowerAssertGradleExtension> {
  functions = listOf("kotlin.assert", "kotlin.test.assertTrue")
}

I wounder if it will work for infix though?

I don't currently have an idea of what might work, but things will surely become clearer with some experimentation. It probably makes sense to do this early up and then provide some feedback to Brian for consideration.

I don't currently have an idea of what might work, but things will surely become clearer with some experimentation. It probably makes sense to do this early up and then provide some feedback to Brian for consideration.

Yep agreed.

• Consider switching default TestIsolationMode to InstancePerTest to better align with the 'expected' default behavior from JUnit

• Consider/Check possibility/ignore gradle limitations on TEST_AND_CONTAINER If you remember the problem, we wanted to merge FreeSpec into StringSpec and some other similar things. The problem was that Gradle doesn't support TEST_AND_CONTAINER correctly, and for such reason we couldn't use that kind of test. We can reconsider if we want to support gradle mainly/add our own plugin or something similar to detach even more from Gradle

• Consider/Check possibility/ignore gradle limitations on TEST_AND_CONTAINER If you remember the problem, we wanted to merge FreeSpec into StringSpec and some other similar things. The problem was that Gradle doesn't support TEST_AND_CONTAINER correctly, and for such reason we couldn't use that kind of test. We can reconsider if we want to support gradle mainly/add our own plugin or something similar to detach even more from Gradle

Gradle still doesn't correctly support TEST_AND_CONTAINER. That ticket has been open since 2017 so unlikely to be fixed.

I re-opened the discussions feature and added a category for 6.0.. https://github.com/kotest/kotest/discussions/categories/6-0-changes

I think the discussions will be easier to follow/participate in if it's 1 thread per change.. what do you think? We can close discussions down again if there's no consensus in favor of this.

+1 for "Update the kotest plugin to run individual tests in native and js". Would be helpful for both intellij and gradle plugins to work nicer with multiplatform projects

What about unifying the base package?

Like, from "com.sksamuel.kotest.listeners.spec.singleinstance" to "io.kotest.listeners.spec.singleinstance".

I looked at the code and only the package of the test code is not unified, so this change will not be a breaking change.

@kshired The reason we keep some tests in com.sksamuel.kotest is usually to test things related to package where we don't want to exclude it. We usually exclude io.kotest.* from anything package-related.

I'm not sure if that's the case for the test you mentioned, but it's very likely to be

Yeah kotest is setup to fold io.kotest stacktraces and things like that, so using my domain for tests is an easy way to avoid that. That should only be used for tests tho.

For breaking changes I think it is right time to upgrade gradle version to support java 17 and 21!

for 6.0 I think this kind of major change is possible!

gradle compatibility matrix

For breaking changes I think it is right time to upgrade gradle version to support java 17 and 21!

for 6.0 I think this kind of major change is possible!

gradle compatibility matrix

If we compile with Java8 does it matter what version of gradle we use ?

@mirageoasis @sksamuel I've created Upgrade gradle version to support java 17 and 21 to track your discussion

@OliverO2 @sksamuel I've created Consider making use of kotlin-power-assert which will become part of the standard compiler plugin set (KT-63607) to track your discussion

maybe adding linter would be great to enforce coding styles! considering code base that kotest has applying linter on major version change would be nice!