`ginkgo labels ./...` doesn't find the labels I added.

[astrieanna]

trafficstars

Context

I just added labels to our integration tests. All the labels are on Describe blocks. For example:

var _ = Describe("Communities Repo Integration", Label("integration"), func() {

I usually use ginkgo -p ./... to run all the tests, and that continues to work perfectly after this change.

What Happened

To confirm that things were working, I tried ginkgo labels ./... from the root folder of my repo. I also tried ginkgo run --label-filter=integration. The output of both commands consistently shows that there are no labels anywhere!

Specifically, ginkgo labels has a line for each package, and each one says No labels found. All 61 of the packages I expected to see are there.

% ginkgo labels ./...
api: No labels found
e2e: No labels found
...
communities: No labels found
...

And ginkgo --dry-run -v --label-filter="integration" ./... shows that no tests will be run in any suite; they're all skipped.

What I expected to happen

When I run ginkgo labels ./... in the root folder of my repository, I expect it to list the suite for each package, and for the packages that I just labeled, it should say integration.

% ginkgo labels ./...
api: No labels found
e2e: No labels found
...
communities: integration
...

And ginkgo --dry-run -v --label-filter="integration" ./... should show that ginkgo would run the tests in the 9 suites that I labelled.

Versions

% ginkgo version
Ginkgo Version 2.1.1

All the ginkgo test files in my codebase import:

. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"

[onsi]

Strange. Any chance your repo is open source so i can take a look?

[onsi]

Also, if you run ginkgo -v in a package that you've added labels to do you see an annotation next to the name of the spec for the ones that are labeled? (it would look like [integration] in your example).

btw thanks for the super clear summary. it's a mystery how this could be happening at this point since everything you've shared sounds like it should be working.

[onsi]

Sorry one last thought. Do things behave differently if you use ginkgo -r instead of ginkgo ./...? i can't imagine that's the issue but it's worth ruling out.

[astrieanna]

Unfortunately, no the repo isn't open-source.

ginkgo -r and ginkgo ./... seems to behave identically.

Examining the output of ginkgo -v in one of the packages was really useful! Our previous way of avoiding integration tests via a build flag was preventing those tests from running -- so I needed to remove a few //go:build integration to see the integration tests run.

With the build flags fixed, ginkgo --label-filter works perfectly. However, ginkgo labels still doesn't work. It doesn't work at the top-level or in a single package. It doesn't work with -r or ./... or (in a package) ..

[onsi]

OK phew. Makes sense about the build tag interfering and I'm glad to hear that --label-filter is working well.

The implementation for ginkgo labels is incredibly naive (it's just a hair more sophisticated than grep). One thing is that it'll only look at *_test.go files. I know some folks organize their specs into separate packages and then import them into a single test suite. Any chance your test suite does that? If so then ginkgo labels currently won't work.

[oliviassss]

@onsi Hi I have a quick question, in order to use --label-filter and --focus-file, which ginkgo version is required as minimal?

[onsi]

hey @oliviassss you need 2.0 at least. The Label decorator and --label-filter are a new features in 2.0

[XiShanYongYe-Chang]

I am using ginkgo 2.1.3 and have had the same problem.