ginkgo
ginkgo copied to clipboard
DeferCleanup() or AfterAll() is not triggered if specs in nested container node is skipped
Hello! I know it is probably not a recommended practice to write tests with so many nested container nodes (and I'm currently rewriting specs in my test to be independent), but I thought it's worth to report this issue anyway.
This is the code
var _ = Describe("Describe level 1", Ordered, func() {
BeforeAll(func() {
log.Println("Executing BeforeAll, level 1")
DeferCleanup(log.Println, "Executing DeferCleanup, level 1")
})
It("It1 level 1", func() {
log.Println("Executing It1 level 1")
Expect(1).To(Equal(1))
})
Describe("Describe level 2", func() {
BeforeAll(func() {
if os.Getenv("SKIP") == "true" {
Skip("Skipping Describe level 2")
}
log.Println("Executing BeforeAll, level 2")
DeferCleanup(log.Println, "Executing DeferCleanup, level 2")
})
It("It1 level 2", func() {
log.Println("Executing It1 level 2")
Expect(1).To(Equal(1))
})
// Describe("Describe level 3", func() {
// BeforeAll(func() {
// log.Println("Executing BeforeAll, level 3")
// DeferCleanup(log.Println, "Executing DeferCleanup, level 3")
// })
// It("It1 level 3", func() {
// log.Println("Executing It1 level 3")
// Expect(1).To(Equal(1))
// })
// })
})
})
When I run the tests with env var SKIP=true
(which results in all specs in container node Describe level 2
to be skipped), the DeferCleanup, level 1
is called, which is expected (see the log 2022/05/18 11:06:30 Executing DeferCleanup, level 1
)
➜ ginkgo SKIP=true ginkgo
Running Suite: Ginkgo Suite - /Users/psturc/learn/go/ginkgo
===========================================================
Random Seed: 1652864789
Will run 2 of 2 specs
•2022/05/18 11:06:30 Executing BeforeAll, level 1
2022/05/18 11:06:30 Executing It1 level 1
------------------------------
S [SKIPPED] [0.000 seconds]
Describe level 1
/Users/psturc/learn/go/ginkgo/main_test.go:11
Describe level 2 [BeforeAll]
/Users/psturc/learn/go/ginkgo/main_test.go:21
It1 level 2
/Users/psturc/learn/go/ginkgo/main_test.go:28
Skipping Describe level 2
In [BeforeAll] at: /Users/psturc/learn/go/ginkgo/main_test.go:23
------------------------------
2022/05/18 11:06:30 Executing DeferCleanup, level 1
Ran 1 of 2 Specs in 0.001 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 1 Skipped
PASS
Ginkgo ran 1 suite in 1.190130184s
Test Suite Passed
However if I uncomment the code for Describe level 3
, and run the test with the same command, the cleanup (DeferCleanup, level 1
) doesn't happen.
➜ ginkgo SKIP=true ginkgo
Running Suite: Ginkgo Suite - /Users/psturc/learn/go/ginkgo
===========================================================
Random Seed: 1652865119
Will run 3 of 3 specs
•2022/05/18 11:12:01 Executing BeforeAll, level 1
2022/05/18 11:12:01 Executing It1 level 1
------------------------------
S [SKIPPED] [0.000 seconds]
Describe level 1
/Users/psturc/learn/go/ginkgo/main_test.go:11
Describe level 2 [BeforeAll]
/Users/psturc/learn/go/ginkgo/main_test.go:21
It1 level 2
/Users/psturc/learn/go/ginkgo/main_test.go:28
Skipping Describe level 2
In [BeforeAll] at: /Users/psturc/learn/go/ginkgo/main_test.go:23
------------------------------
S [SKIPPED] [0.000 seconds]
Describe level 1
/Users/psturc/learn/go/ginkgo/main_test.go:11
Describe level 2
/Users/psturc/learn/go/ginkgo/main_test.go:20
Describe level 3
/Users/psturc/learn/go/ginkgo/main_test.go:32
[It] It1 level 3
/Users/psturc/learn/go/ginkgo/main_test.go:37
Spec skipped because Skip() was called in BeforeAll
In [It] at: /Users/psturc/learn/go/ginkgo/main_test.go:37
------------------------------
Ran 1 of 3 Specs in 0.001 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 2 Skipped
PASS
Ginkgo ran 1 suite in 1.892848673s
Test Suite Passed
It happens only in case the specs in second container node (Describe level 2
) are skipped. (If I run the tests with SKIP=false ginkgo
, the cleanup works as expected).
hey @psturc thanks for reporting this. Looks like it might be an edge-case bug that you've found. I'll dig into it and see how if I can push a fix out soon.
Hi @onsi did you get a chance to look into this (no pressure, just curious :)?
hey sorry, no. i've been super behind!