go icon indicating copy to clipboard operation
go copied to clipboard

Published 20 hours ago • verified publisher icon golang

x/tools/gopls: gopls implementation result does not include all types in vendor directory

Open x-yuri opened this issue 2 years ago • 8 comments

What version of Go, VS Code & VS Code Go extension are you using?

Version Information
  • Run go version to get version of Go from the VS Code integrated terminal.
    • go version go1.19.1 linux/amd64
  • Run gopls -v version to get version of Gopls from the VS Code integrated terminal. 
    Build info
----------
golang.org/x/tools/gopls v0.9.5
    golang.org/x/tools/gopls@(devel)
    github.com/BurntSushi/[email protected] h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
    github.com/google/[email protected] h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
    github.com/sergi/[email protected] h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
    golang.org/x/[email protected] h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
    golang.org/x/exp/[email protected] h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
    golang.org/x/[email protected] h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
    golang.org/x/[email protected] h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
    golang.org/x/[email protected] h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
    golang.org/x/[email protected] h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
    golang.org/x/[email protected] h1:glzimF7qHZuKVEiMbE7UqBu44MyTjt5u6j3Jz+rfMRM=
    golang.org/x/[email protected] h1:YnB27EXBD8XxB0JcaOeluuvhF2kS4DrH0k+lpopG2xc=
    honnef.co/go/[email protected] h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
    mvdan.cc/[email protected] h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
    mvdan.cc/xurls/[email protected] h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
go: go1.19.1
  • Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders. 
    1.71.2
74b1f979648cc44d385a2286793c226e611f59e7
x64
  • Check your installed extensions to get the version of the VS Code Go extension
    • v0.35.2
  • Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command. 
    Checking configured tools....
GOBIN: undefined
toolsGopath: 
gopath: /home/yuri/go
GOROOT: /usr/lib/go
PATH: /home/yuri/perl5/bin:/home/yuri/bin:/home/yuri/Dropbox/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

    go: /usr/bin/go: go version go1.19.1 linux/amd64

    gotests:    not installed
    gomodifytags:   not installed
    impl:   not installed
    goplay: not installed
    dlv:    /usr/bin/dlv    (version: (devel) built with go: go1.19)
    staticcheck:    /usr/bin/staticcheck    (version: (devel) built with go: go1.18.5)
    gopls:  /usr/bin/gopls  (version: (devel) built with go: go1.19.1)

go env
Workspace Folder (docker): /home/yuri/go/src/github.com/docker/docker
    GO111MODULE=""
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/home/yuri/.cache/go-build"
    GOENV="/home/yuri/.config/go/env"
    GOEXE=""
    GOEXPERIMENT=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOINSECURE=""
    GOMODCACHE="/home/yuri/go/pkg/mod"
    GONOPROXY=""
    GONOSUMDB=""
    GOOS="linux"
    GOPATH="/home/yuri/go"
    GOPRIVATE=""
    GOPROXY="https://proxy.golang.org,direct"
    GOROOT="/usr/lib/go"
    GOSUMDB="sum.golang.org"
    GOTMPDIR=""
    GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
    GOVCS=""
    GOVERSION="go1.19.1"
    GCCGO="gccgo"
    GOAMD64="v1"
    AR="ar"
    CC="gcc"
    CXX="g++"
    CGO_ENABLED="1"
    GOMOD="/dev/null"
    GOWORK=""
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1841263837=/tmp/go-build -gno-record-gcc-switches"

Share the Go related settings you have added/edited

I didn't change any settings.

Describe the bug

I trying to examine the moby project, and the extension seems to ignore the vendor dir. Possibly because there's no go.mod at the root of the repository. E.g. it can't find the implementation of p.repo.Blobs, that is located here. As long as I need to navigate within the main module it seems to work fine.

Steps to reproduce the behavior:

  1. Clone https://github.com/moby/moby to ~/go/src/github.com/docker/docker.
  2. Open the directory in code, then distribution/pull_v2.go (line 903).
  3. Right click on Blobs, choose "Go to Implementations".

You'll see only some mock implementations from distribution/push_v2_test.go.

Screenshots or recordings

x-yuri avatar Oct 04 '22 04:10 x-yuri

Can you try with the gopls at master?

git clone https://go.googlesource.com/tools
cd tools/gopls
go install

And restart the gopls (Go: Restart Language Server) or reload the vscode window.

I could reproduce the issue with the released version of gopls. But I see the gopls built from master now finds code in the vendor location correctly.

hyangah avatar Oct 06 '22 02:10 hyangah

I've just tried, but see no difference. It can't find that particular implementation in the vendor dir. I tried both restarting the server and code.

$ ~/go/bin/gopls version
golang.org/x/tools/gopls master
    golang.org/x/tools/gopls@(devel)

Is there a way to make a curl request to gopls or something to confirm this? Can you possibly tell what's the extension's part in it? Does it simply tell code how to connect to the server? Or there's more to it? Any links to the code?

x-yuri avatar Oct 06 '22 15:10 x-yuri

Could you please collect a trace and share it here?

You can do this by setting "go.languageServerFlags": ["-rpc.trace"] in your settings.json and then sharing the logs you see in the output channel:

Screen Shot 2021-01-27 at 2 53 49 PM

suzmue avatar Oct 06 '22 17:10 suzmue

To check the detailed version of gopls -

$ go version -m  /usr/bin/gopls

The gopls your extension is using is printed in the Go: Locate Configured Go Tools output.

hyangah avatar Oct 06 '22 19:10 hyangah

The trace: https://gist.github.com/x-yuri/985f8289b15e07fe1038c1eafe679b0b

$ go version -m  /usr/bin/gopls 
/usr/bin/gopls: go1.19.1
	path	golang.org/x/tools/gopls
	mod	golang.org/x/tools/gopls	(devel)	
	dep	github.com/BurntSushi/toml	v1.2.0	h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
	dep	github.com/google/go-cmp	v0.5.8	h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
	dep	github.com/sergi/go-diff	v1.1.0	h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
	dep	golang.org/x/exp	v0.0.0-20220722155223-a9213eeb770e	h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
	dep	golang.org/x/exp/typeparams	v0.0.0-20220722155223-a9213eeb770e	h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
	dep	golang.org/x/mod	v0.6.0-dev.0.20220419223038-86c51ed26bb4	h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
	dep	golang.org/x/sync	v0.0.0-20220722155255-886fb9371eb4	h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
	dep	golang.org/x/sys	v0.0.0-20220722155257-8c9f86f7a55f	h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
	dep	golang.org/x/text	v0.3.7	h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
	dep	golang.org/x/tools	v0.1.13-0.20220908144252-ce397412b6a4	h1:glzimF7qHZuKVEiMbE7UqBu44MyTjt5u6j3Jz+rfMRM=
	dep	golang.org/x/vuln	v0.0.0-20220901221904-62b0186a1058	h1:YnB27EXBD8XxB0JcaOeluuvhF2kS4DrH0k+lpopG2xc=
	dep	honnef.co/go/tools	v0.3.2	h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
	dep	mvdan.cc/gofumpt	v0.3.1	h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
	dep	mvdan.cc/xurls/v2	v2.4.0	h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
	build	-compiler=gc
	build	-trimpath=true
	build	CGO_ENABLED=1
	build	GOARCH=amd64
	build	GOOS=linux
	build	GOAMD64=v1

$ go version -m  ~/go/bin/gopls
/home/yuri/go/bin/gopls: go1.19.1
	path	golang.org/x/tools/gopls
	mod	golang.org/x/tools/gopls	(devel)	
	dep	github.com/BurntSushi/toml	v1.2.0	h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
	dep	github.com/google/go-cmp	v0.5.8	h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
	dep	github.com/sergi/go-diff	v1.1.0	h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
	dep	golang.org/x/exp	v0.0.0-20220722155223-a9213eeb770e	h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
	dep	golang.org/x/exp/typeparams	v0.0.0-20220722155223-a9213eeb770e	h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
	dep	golang.org/x/mod	v0.6.0-dev.0.20220419223038-86c51ed26bb4	h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
	dep	golang.org/x/sync	v0.0.0-20220722155255-886fb9371eb4	h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
	dep	golang.org/x/sys	v0.0.0-20220808155132-1c4a2a72c664	h1:v1W7bwXHsnLLloWYTVEdvGvA7BHMeBYsPcF0GLDxIRs=
	dep	golang.org/x/text	v0.3.7	h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
	dep	golang.org/x/tools	v0.1.13-0.20220928184430-f80e98464e27
	=>	../	(devel)	
	
	dep	golang.org/x/vuln	v0.0.0-20221004232641-2aa0553d353b	h1:8Tu9pgIV7kt8ulNtzidzpLl9E9l1i+U4QLdKG0ZzHyE=
	dep	honnef.co/go/tools	v0.3.3	h1:oDx7VAwstgpYpb3wv0oxiZlxY+foCpRAwY7Vk6XpAgA=
	dep	mvdan.cc/gofumpt	v0.3.1	h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
	dep	mvdan.cc/xurls/v2	v2.4.0	h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
	build	-compiler=gc
	build	CGO_ENABLED=1
	build	CGO_CFLAGS=
	build	CGO_CPPFLAGS=
	build	CGO_CXXFLAGS=
	build	CGO_LDFLAGS=
	build	GOARCH=amd64
	build	GOOS=linux
	build	GOAMD64=v1
	build	vcs=git
	build	vcs.revision=bd8c28ff5c76b957a23f93a67337acc0c5a4f67e
	build	vcs.time=2022-10-06T14:25:42Z
	build	vcs.modified=false

Can you try with the gopls at master?

Now that I think about it, maybe I should uninstall gopls and try again.

x-yuri avatar Oct 06 '22 20:10 x-yuri

But Go: Locate Configured Go Tools says:

gopls: /home/yuri/go/bin/gopls (version: (devel) built with go: go1.19.1)

All in all looks like a gopls issue. Should I report it there?

x-yuri avatar Oct 06 '22 20:10 x-yuri

Thanks for capturing the log.

I also verified this is reproducible - sorry that I was confused by "Find all references" reporting the interface definition correctly. From your original report, I see that you are reporting that the "Find all implementations" (or Implementations) result is missing a type in the vendor directory.

In the example you shared, p.repo.Blobs implements a method of the interface distribution.Repository, so you wanted the implementations query to return all implementations found not only in the main workspace, including distribution.repository. (not distribution.blobs.Get). Here what I mean by main workspace is the code excluding vendor or third-party dependencies.

I think this happens because the implementation type (distribution.repository) in vendor is unexported. For performance, gopls employs various heuristics; one of them is to avoid full type-checking when a package is outside the workspace (i.e. packages in dependency). (relevant code: workspaceParseMode). Packages under vendor seem to be considered packages outside workspace, then gopls will operate in exported symbol only mode. This can affect the implementations query search space and results in skipping unexported types.

I verified gopls returns the implementation type if I rename distribution.repository to an exported name. (See the one with red underline)

Screen Shot 2022-10-11 at 8 44 36 PM

I will transfer this to gopls issue tracker, but I am afraid this is subtle and may not be fixable easily.

hyangah avatar Oct 12 '22 01:10 hyangah

Oh, I just noticed that one of the links in my report is wrong (the second one):

E.g. it can't find the implementation of p.repo.Blobs, that is located here.

And that's probably what you meant here:

In the example you shared, p.repo.Blobs implements a method of the interface distribution.Repository, so you wanted the implementations query to return all implementations found not only in the main workspace, including distribution.repository. (not distribution.blobs.Get).

So to make sure we're on the same page...

I know that p.repo.Blobs() here (distribution/pull_v2.go, v2Puller.pullSchema2Config()) invokes repository.Blobs() here (vendor/github.com/docker/distribution/registry/client/repository.go). According to vscode p.repo (from v2Puller.pullSchema2Config()) implements the Repository interface from vendor/github.com/docker/distribution/registry.go.

In the same vein blobs.Get() here (distribution/pull_v2.go, v2Puller.pullSchema2Config()) invokes blobs.Get() here (vendor/github.com/docker/distribution/registry/client/repository.go). According to vscode blobs (from v2Puller.pullSchema2Config()) implements the BlobProvider interface from vendor/github.com/docker/distribution/blobs.go.

x-yuri avatar Oct 12 '22 12:10 x-yuri

Change https://go.dev/cl/454637 mentions this issue: gopls/internal/regtest/misc: test Implementations + vendor

gopherbot avatar Dec 01 '22 22:12 gopherbot

I am afraid this isn't fixed. I still don't see the implementation in vendor. From my initial analysis in https://github.com/golang/go/issues/56169#issuecomment-1275455319, I thought the root cause was the handling of unexported types outside the main module (regardless it's vendored or not).

hyangah avatar Dec 02 '22 22:12 hyangah

Reopen for continued investigation & fix verification. cc @adonovan

hyangah avatar Dec 05 '22 17:12 hyangah

It looks like the issue may be related to file watching: gopls doesn't ask VS Code to watch the vendor directory. (I discovered this after implementing proper support for glob patterns in our fake LSP client).

findleyr avatar Jan 10 '23 03:01 findleyr

I cloned moby into $(go env GOPATH)/src and was able to build it (with GO111MODULE=auto) and use gopls@latest within it. In particular, the implementations operation applied to the p.repo.Blobs call in pull_v2.go reported three results:

distribution/push_v2_test.go
559:func (m *mockRepoWithBlob) Blobs(ctx context.Context) distribution.BlobStore {
652:func (m *mockRepo) Blobs(ctx context.Context) distribution.BlobStore {
vendor/github.com/docker/distribution/registry/client/repository.go
161:func (r *repository) Blobs(ctx context.Context) distribution.BlobStore {

I confirm that with v0.11.0 only the first two were reported, so this issue appears to have been fixed.

adonovan avatar May 23 '23 15:05 adonovan