Test behavior breaks when testing v2+ gomodules

[ralf-cestusio]

When following the guidelines for writing v2+ packages from [Go blog] (https://blog.golang.org/v2-go-modules#TOC_4.) executing ginkgo ./... in the v1 version starts throwing error messages looking like

Failed to compile entities: main module (go.mycompany.com/services/foo) does not contain package go.mycompany.com/services/foo/v2/

commands in the standard go tool chain (including go test ./...) do not traverse into go submodules and are therefore avoiding the change of go module.

[blgm]

Hi @magicmoose. That's curious.

In order to have something concrete to talk about, I've created this repo: https://github.com/blgm/foo

It has "v1" code at the top level and "v2" in a directory - as per the blog. It seems to work for me. I'm guessing that there's a difference between what we are doing, and understanding that difference will move the issue forward. Does this repo look like what you're doing? Are there any differences that stand out?

[ralf-cestusio]

you are missing the go.mod file in the v2 directory. so the step from the blog looking like

$ cp go.mod v2/go.mod
$ go mod edit -module github.com/googleapis/gax-go/v2 v2/go.mod

[blgm]

Got it! I can reproduce the issue you describe:

$ ginkgo ./...
[1619723369] Actor Suite - 1/1 specs v1
• SUCCESS! 268.133µs PASS
Failed to compile actor:

main module (github.com/blgm/foo) does not contain package github.com/blgm/foo/v2/actor

Ginkgo ran 2 suites in 2.84513825s
Test Suite Failed

$ ginkgo -r
[1619723386] Actor Suite - 1/1 specs v1
• SUCCESS! 256.338µs PASS
Failed to compile actor:

main module (github.com/blgm/foo) does not contain package github.com/blgm/foo/v2/actor

Ginkgo ran 2 suites in 3.388334365s
Test Suite Failed

But go test works by skipping the Go Module within Go Module:

$ go test ./...
?   	github.com/blgm/foo	[no test files]
ok  	github.com/blgm/foo/actor	(cached)

[onsi]

I started looking through the go source code to figure out how go test computes packages for ./... and.... there's a fair bit of complexity there.

I see two options:

• continue to play whackamole and implement logic to prevent ginkgo from traversing this edge case.
• have ginkgo call go list to faithfully reproduce the go toolchain's behavior. the downside here is the performance hit of invoking go list - it doesn't look like I can pull in the underlying logic/library as it's all locked away in go's internal packages.

[blgm]

The advantage of using go list is we delegate complexity and maintain exactly the same behaviour as go test. I wonder how noticeable the effect is of staring another process?

[onsi]

I'm gonna try it out over on the v2 branch. On my admittedly crappy 2012 macbook pro it takes about a half second to run.

[onsi]

But yes. Agreed that it's the more sustainable long term solution.

[ralf-cestusio]

I would agree that it is the best long term solution. The go team has changed their mind about how to deal with modules and module versions multiple times. Using their own tool chain would probably remove the need to touch this again in a few months.

[onsi]

Ok sounds good all around. I'll work on using go list - though i'm only planning on making the change in v2 (which i'm hoping will ship in the next couple months).

[ralf-cestusio]

Sounds good! Its a good driving factor to get ppl to use v2 when it comes out.

The downside of this is that i will have to temporarily move our ci pipelines at work to a custom patched ginkgo version since our pipeline fails since we introduced a v2. I will post my temp fix(when its done) in case there will be interest in mergeing this to v1.

[onsi]

that would be super super helpful, actually. thanks!

[onsi]

i'm mostly wanting to spend most of my cycles on v2 - but will happily merge a working patch into v1 (and borrow from it to accelerate the v2 side too!)

[ralf-cestusio]

i have a poc in https://github.com/magicmoose/ginkgo/tree/no_module_cross but realised that i was using a go 1.16+ feature. I will have to find some time to rewrite that against 1.15. But feel free to take a look to see if you like the general approach.

[onsi]

Sorry for the radio silence. Am still planning to leverage go list in V2 but haven't gotten to it yet.

[onsi]

I've finally managed to circle back to this as I near the finish line on ginkgo v2...

...and it looks like this issue is "fixed" in Ginkgo V2. I've fetched @blgm's aptly named https://github.com/blgm/foo repo and, at least on v2 of Ginkgo, I'm seeing:

ginkgo -r
[1633550837] Actor Suite - 1/1 specs v1
• SUCCESS! 362.961µs PASS
[1633550837] Actor Suite - 1/1 specs v2
• SUCCESS! 217.385µs PASS

When I run with Ginkgo V1 I see the compilation issue y'all were seeing. There have been very many changes in V2 and I bet some of the cleanup has led to this getting resolved.

With that said, I do see go test ./... at the top level only run the v1 tests and go test ./... within v2 only run the v2 tests. I appreciate that's the go toolchain's default behavior though I'm a bit confused by it ("I thought I told you to run all the tests in this repo"). Honestly, I'm inclined to leave sleeping dragons lie and ship with this behavior but am open to feedback to the contrary!

[ralf-cestusio]

I totally missed this message. I will go give it a try in the (hopefully) near future. I think the behavior you are seeing is the now default go behavior: no command passes module barriers. Its the same for go build ./... (this did bite me as well since running this in the root does not mean you can compile v2)

[onsi]

Ima gonna go ahead and close this but please feel free to reopen if need be