cloud-builders-community icon indicating copy to clipboard operation
cloud-builders-community copied to clipboard

Windows builder slow file copy

Open jbrekelmans opened this issue 6 years ago • 0 comments

Affected builder image

gcr.io/cloud-builders-community/windows

Expected Behavior

When the builder copies files from the workspace on the Cloud Build agent to the Windows VM (based on Windows Remote Management protocol), it should quickly copy a large workspace of 2000 directories and 100 MB of data. It should take a few minutes.

Actual Behavior

The build times out after 2 hours of trying to copy data.

Steps to Reproduce the Problem

N/A

Additional Info

The package used to copy files is slow: https://github.com/GoogleCloudPlatform/cloud-builders-community/blob/master/windows-builder/builder/builder/remote.go#L12 It sequentially processes files, breaks each file into 8KB chunks, and performs 3 HTTP round trips for each such chunk. This implementation of WinRM-based file copying is hundreds of times faster: https://github.com/jbrekelmans/winrm (disclaimer I am the author).

jbrekelmans avatar Jan 14 '20 08:01 jbrekelmans