buildkit icon indicating copy to clipboard operation
buildkit copied to clipboard

ci(dockerd): trigger through chat-ops

Open crazy-max opened this issue 2 years ago • 3 comments

While checking https://github.com/moby/moby/pull/43661 I encounter an issue with our dockerd workflow and some recent tests added here fail against this worker.

I changed the behavior of the workflow to add a trigger through comment in PR (chat-ops): https://github.com/crazy-max/buildkit/pull/8

# will trigger dockerd workflow with default moby version (20.10.18 atm)
/test-dockerd
# download and test 20.10.17
/test-dockerd 20.10.17
# build master branch and test
/test-dockerd https://github.com/moby/moby.git#master

I skip the tests in the meantime but we might be able to make them compatible with dockerd worker (cc @jedevc @tonistiigi @deitch):

=== CONT  TestIntegration/TestExportAnnotations/worker=dockerd
    client_test.go:6132: 
        	Error Trace:	client_test.go:6132
        	            				run.go:85
        	            				run.go:184
        	Error:      	Received unexpected error:
        	            	exporter "image" could not be found
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:196
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/api/services/control.(*controlClient).Solve
        	            		/src/api/services/control/control.pb.go:1535
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:231
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	            	failed to solve
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:244
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	Test:       	TestIntegration/TestExportAnnotations/worker=dockerd
=== CONT  TestIntegration/TestCallInfo/worker=dockerd
    client_test.go:6039: 
        	Error Trace:	client_test.go:6039
        	            				run.go:85
        	            				run.go:184
        	Error:      	Received unexpected error:
        	            	Unimplemented: unknown method Info for service moby.buildkit.v1.Control
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:196
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/api/services/control.(*controlClient).Info
        	            		/src/api/services/control/control.pb.go:1616
        	            	github.com/moby/buildkit/client.(*Client).Info
        	            		/src/client/info.go:22
        	            	github.com/moby/buildkit/client.testCallInfo
        	            		/src/client/client_test.go:6038
        	            	github.com/moby/buildkit/util/testutil/integration.testFunc.Run
        	            		/src/util/testutil/integration/run.go:85
        	            	github.com/moby/buildkit/util/testutil/integration.Run.func2.1
        	            		/src/util/testutil/integration/run.go:184
        	            	testing.tRunner
        	            		/usr/local/go/src/testing/testing.go:1439
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	            	failed to call info
        	            	github.com/moby/buildkit/client.(*Client).Info
        	            		/src/client/info.go:24
        	            	github.com/moby/buildkit/client.testCallInfo
        	            		/src/client/client_test.go:6038
        	            	github.com/moby/buildkit/util/testutil/integration.testFunc.Run
        	            		/src/util/testutil/integration/run.go:85
        	            	github.com/moby/buildkit/util/testutil/integration.Run.func2.1
        	            		/src/util/testutil/integration/run.go:184
        	            	testing.tRunner
        	            		/usr/local/go/src/testing/testing.go:1439
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	Test:       	TestIntegration/TestCallInfo/worker=dockerd
=== CONT  TestIntegration/TestOCILayoutPlatformSource/worker=dockerd
    client_test.go:1733: 
        	Error Trace:	client_test.go:1733
        	            				run.go:85
        	            				run.go:184
        	Error:      	Received unexpected error:
        	            	exporter "oci" could not be found
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:196
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/api/services/control.(*controlClient).Solve
        	            		/src/api/services/control/control.pb.go:1535
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:231
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	            	failed to solve
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:244
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	Test:       	TestIntegration/TestOCILayoutPlatformSource/worker=dockerd
=== CONT  TestIntegration/TestOCILayoutSource/worker=dockerd
    client_test.go:1603: 
        	Error Trace:	client_test.go:1603
        	            				run.go:85
        	            				run.go:184
        	Error:      	Received unexpected error:
        	            	exporter "oci" could not be found
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:196
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/api/services/control.(*controlClient).Solve
        	            		/src/api/services/control/control.pb.go:1535
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:231
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	            	failed to solve
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:244
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	Test:       	TestIntegration/TestOCILayoutSource/worker=dockerd
=== CONT  TestIntegration/TestNamedOCILayoutContext/worker=dockerd/frontend=client
    dockerfile_test.go:5379: 
        	Error Trace:	dockerfile_test.go:5379
        	            				run.go:85
        	            				run.go:184
        	Error:      	Received unexpected error:
        	            	exporter "oci" could not be found
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:196
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/api/services/control.(*controlClient).Solve
        	            		/src/api/services/control/control.pb.go:1535
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:231
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	            	failed to solve
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:244
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	Test:       	TestIntegration/TestNamedOCILayoutContext/worker=dockerd/frontend=client
=== CONT  TestIntegration/TestNamedImageContextTimestamps/worker=dockerd/frontend=client
2022/08/05 01:40:21 http2: server connection error from localhost: connection error: PROTOCOL_ERROR
    dockerfile_test.go:5230: 
        	Error Trace:	dockerfile_test.go:5230
        	            				run.go:85
        	            				run.go:184
        	Error:      	Received unexpected error:
        	            	exporter "image" could not be found
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:196
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/api/services/control.(*controlClient).Solve
        	            		/src/api/services/control/control.pb.go:1535
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:231
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	            	failed to solve
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:244
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:57
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1571
        	Test:       	TestIntegration/TestNamedImageContextTimestamps/worker=dockerd/frontend=client

crazy-max avatar Aug 05 '22 02:08 crazy-max

Only one I'm not sure about is the TestCallInfo: Unimplemented: unknown method Info for service moby.buildkit.v1.Control - is that moby's buildkit worker not implementing one of the methods? If so, maybe we should have an explicit check that moby's error message is unimplemented, so that we remember to re-enable it once moby re-vendors buildkit?

As it's only on SkipIfDockerd with message not implemented I think it's enough no?: https://github.com/moby/buildkit/runs/7684232245?check_suite_focus=true#step:9:1201

=== CONT  TestIntegration/TestCallInfo/worker=dockerd
    client_test.go:6037: dockerd worker can not currently run this test due to missing features (not implemented)

Also yes I think I will open a PR on moby just to add the implementation and backport to 22.06 and maybe 20.10 (cc @thaJeztah)

Edit: BuildKit 0.10.3 on Moby does not have https://github.com/moby/buildkit/pull/2725 so we might need to wait for 0.11 or we could backport to v0.10 branch if you are ok with it @tonistiigi?

crazy-max avatar Aug 05 '22 12:08 crazy-max

Running this on pull requests doesn't really work because the changes a pull request makes are not in the dockerd worker until PR gets merged and vendored into dockerd.

tonistiigi avatar Aug 06 '22 00:08 tonistiigi

Changed the behavior of the workflow to add a trigger through comment in PR (chat-ops): https://github.com/crazy-max/buildkit/pull/8

# will trigger dockerd workflow with default moby version (20.10.18 atm)
/test-dockerd
# download and test 20.10.17
/test-dockerd 20.10.17
# build master branch and test
/test-dockerd https://github.com/moby/moby.git#master

Edit: Actually the checked out ref does not look right.

crazy-max avatar Sep 09 '22 16:09 crazy-max