Enable WCOW custom frontends to communicate with BuildKit via named pipe bridge

[billywr]

PR adds support for custom frontends in WCOW by enabling gRPC communication from the container child process to the BuildKit in host machine via a named pipe.

Currently, the custom frontend bridge uses stdio file descriptors (FDs), which work well for Linux container child processes and also work in Windows containers. However, they are not supported for communication between WCOW container 'child' processes and the host machine process. As a result, custom frontends running in Windows containers are unable to communicate with the host machine's BuildKit instance

This change:

• Introduces a named pipe (\\.\pipe\buildkit-frontend-bridge) specifically for WCOW custom frontend containers.
• Mounts the pipe into the container .

Fixes #4892

Guide to test

STEP 1. Compile frontend binary You can use go code in this location: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/cmd/dockerfile-frontend/main.go use go build -o ... to create dockerfile-frontend.exe

STEP 2. Build frontend image and push it your docker hub The dockerfile contents to build the frontend image:

FROM mcr.microsoft.com/windows/nanoserver:ltsc2022
LABEL moby.buildkit.frontend.network.none="true"
LABEL moby.buildkit.frontend.caps="moby.buildkit.frontend.inputs,moby.buildkit.frontend.subrequests,moby.buildkit.frontend.contexts"

COPY dockerfile-frontend.exe C:\dockerfile-frontend.exe

ENTRYPOINT ["C:\\dockerfile-frontend.exe"]

Can use this buildctl command to build frontend image

buildctl build `
--frontend=dockerfile.v0 `
--local context="path to your build context" `
--local dockerfile="path to your dockefile" `
--output type=image,name=yourdockerhuburl/dockerfrontend:latest,push=true `
--no-cache 

STEP 3. Use the frontend image in your docker builds

say you have a dockerfile below, notice the use of #syntax to make use of your custom dockerfile frontend

yourdockerhuburl/dockerfrontend:latest comes from the image you pushed in step 2

# syntax=yourdockerhuburl/dockerfrontend:latest

FROM mcr.microsoft.com/windows/nanoserver:ltsc2022
RUN echo hello from WCOW custom frontend

[billywr]

What's the reason stdio FDs would not work for wcow? It doesn't have any dependency on UNIX sockets, just that containers have stdin and stdout capability.

I encountered the error Unavailable: connection error: desc = transport: failed to write client preface: write /dev/stdout: file already closed before configuring a named pipe. Notably, /dev/stdout is not a valid stdio path in Windows, where STDIO (STDIN, STDOUT, STDERR) is managed as handles rather than files. While /dev/stdout could be an abstraction for the STDOUT handle in WCOW, no failures were expected. However, the error persisted, and there is no direct evidence suggesting a premature exit of the frontend executable (dockerfile-frontend.exe) caused it.

The successful resolution with a named pipe without modifying the frontend code strongly indicates that STDIO is unreliable in WCOW buildkit custom frontend scenarios[I have to get better reasons why that is the case, but one possibility could be the container child process might not have inherited the STDOUT handle].

stdout stdin & stderror work just fine like in the case where WCOW container is launched in interactive mode.

[tonistiigi]

Notably, /dev/stdout is not a valid stdio path in

Why would this matter? Where do we open /dev/stdout as path from filesystem. Are you saying os.Stdout does not work on windows? Do you have a trace for that error?

[billywr]

Notably, /dev/stdout is not a valid stdio path in

Why would this matter? Where do we open /dev/stdout as path from filesystem. Are you saying os.Stdout does not work on windows? Do you have a trace for that error?

1. main.main
   buildkit/frontend/dockerfile/cmd/dockerfile-frontend/main.go:30
   │
   └──> 2. github.com/moby/buildkit/frontend/gateway/grpcclient.RunFromEnvironment
        buildkit/frontend/gateway/grpcclient/client.go:100
        │
        └──> 3. github.com/moby/buildkit/frontend/gateway/grpcclient.New
             buildkit/frontend/gateway/grpcclient/client.go:49
             │
             └──> 4.github.com/moby/buildkit/frontend/gateway/pb.(*LLBBridgeClient).Ping
                  buildkit/frontend/gateway/pb/gateway_grpc.pb.go:148
                  │
                  └──> 5. google.golang.org/grpc.(*ClientConn).Invoke
                       buildkit/vendor/google.golang.org/grpc/call.go:35
                       │
                       └──> Error: "transport: failed to write client preface: write /dev/stdout: file already closed