box
box copied to clipboard
box-builder
Google Cloud Builder
Google Cloud Builder would be a good candidate for fully-remote builder/executor implementation, it supports async and parallel modes, but it's probably easier to start with a serial implementation. I'm still learning how GCB works in a another project, will update once I have more info.
Do you have a link to the protocol for future reference? I love this idea but want to make sure it's a thing we can actually do.
Do you have a link to the protocol for future reference? I love this idea but want to make sure it's a thing we can actually do.
I'm not sure how we could drive the API exactly, and how it'd would fit into current vocabulary of box, but perhaps something like "run a block on the remote builder" would be a meaningful first experiment.
I've just tried something very simple and it worked.
> cat cloudbuild.yaml
steps:
- name: 'erikh/box:master'
args: [ 'test.rb' ]
images:
- 'gcr.io/$PROJECT_ID/boxtest'
> cat test.rb
from "golang:1.8.0-alpine"
run "apk update && apk add gcc"
tag "gcr.io/box-demo-159311/boxtest"
> gcloud container builds submit --config cloudbuild.yaml .
Creating temporary tarball archive of 13420 file(s) totalling 278.2 MiB before compression.
Uploading tarball of [.] to [gs://box-demo-159311_cloudbuild/source/1487686963.89.tgz]
Created [https://cloudbuild.googleapis.com/v1/projects/box-demo-159311/builds/12b13592-09d6-4909-8476-38269b268e4b].
Logs are permanently available at [https://console.developers.google.com/logs/viewer?resource=build&project=box-demo-159311&filters=text:12b13592-09d6-4909-8476-38269b268e4b].
------------------------------------------------------------------------------------------------------------ REMOTE BUILD OUTPUT -------------------------------------------------------------------------------------------------------------
starting build "12b13592-09d6-4909-8476-38269b268e4b"
FETCHSOURCE
Fetching storage object: gs://box-demo-159311_cloudbuild/source/1487686963.89.tgz#1487690631416270
Copying gs://box-demo-159311_cloudbuild/source/1487686963.89.tgz#1487690631416270...
Downloading file:///tmp/source-archive.tgz: 70.85 MiB/70.85 MiB
BUILD
Pulling image: erikh/box:master
master: Pulling from erikh/box
75a822cd7888: Pulling fs layer
2ecca30ff69a: Pulling fs layer
2ecca30ff69a: Verifying Checksum
2ecca30ff69a: Download complete
75a822cd7888: Verifying Checksum
75a822cd7888: Download complete
75a822cd7888: Pull complete
2ecca30ff69a: Pull complete
Digest: sha256:79ce6bad6692c6f383d28e021b88f74e358ecc90a27a2f2a3af31cfd2e772bad
Status: Downloaded newer image for erikh/box:master
[test.rb] +++ Execute: from golang:1.8.0-alpine
Pulling "golang:1.8.0-alpine"...done.
[test.rb] +++ Execute: run apk update && apk add gcc
[test.rb] ------ BEGIN OUTPUT ------
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
v3.5.1-38-g3e6cf4c978 [http://dl-cdn.alpinelinux.org/alpine/v3.5/main]
v3.5.1-37-gb1ebb618a7 [http://dl-cdn.alpinelinux.org/alpine/v3.5/community]
OK: 7956 distinct packages available
(1/12) Installing binutils-libs (2.27-r0)
(2/12) Installing binutils (2.27-r0)
(3/12) Installing gmp (6.1.1-r0)
(4/12) Installing isl (0.17.1-r0)
(5/12) Installing libgomp (6.2.1-r1)
(6/12) Installing libatomic (6.2.1-r1)
(7/12) Installing pkgconf (1.0.2-r0)
(8/12) Installing libgcc (6.2.1-r1)
(9/12) Installing mpfr3 (3.1.5-r0)
(10/12) Installing mpc1 (1.0.3-r0)
(11/12) Installing libstdc++ (6.2.1-r1)
(12/12) Installing gcc (6.2.1-r1)
Executing busybox-1.25.1-r0.trigger
OK: 87 MiB in 24 packages
[test.rb] ------- END OUTPUT -------
[test.rb] +++ Execute: tag gcr.io/box-demo-159311/boxtest
[test.rb] +++ Tagged: gcr.io/box-demo-159311/boxtest
[test.rb] +++ Eval Response: sha256:e04f6ab93a07c34bc957abb2a4718a0a9a613f8f659c0e6c2e83affdd651f7e6
[test.rb] +++ Finish: e04f6ab93a07c34bc957abb2a4718a0a9a613f8f659c0e6c2e83affdd651f7e6
PUSH
Pushing gcr.io/box-demo-159311/boxtest
The push refers to a repository [gcr.io/box-demo-159311/boxtest]
05971ceff39e: Preparing
d2d9bee4e0a8: Preparing
dddc627ce7af: Preparing
52546b4d6157: Preparing
c964b132b04e: Preparing
8be01aa5187e: Preparing
60ab55d3379d: Preparing
8be01aa5187e: Waiting
60ab55d3379d: Waiting
c964b132b04e: Pushed
dddc627ce7af: Pushed
d2d9bee4e0a8: Pushed
60ab55d3379d: Mounted from cloud-builders/metadata
8be01aa5187e: Pushed
05971ceff39e: Pushed
52546b4d6157: Pushed
latest: digest: sha256:f8b79261e8b4a70198106c264016ba80b792090cf9ce2c8c965780046514c4eb size: 1784
DONE
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ID CREATE_TIME DURATION SOURCE IMAGES STATUS
12b13592-09d6-4909-8476-38269b268e4b 2017-02-21T15:23:52+00:00 1M24S gs://box-demo-159311_cloudbuild/source/1487686963.89.tgz gcr.io/box-demo-159311/boxtest SUCCESS
>
Perhaps expressing YAML config in Ruby would be a thing, then make request to their API to create a build.
ye ah I don't think it'd be hard to generate instructions instead of executing them. Box is built to handle stuff like that fairly well.
Would you be willing to do this patch? I can assist with any code reorg required.
Erik, I would love to do this, not entirely sure when I will have time for it..
On Wed, 22 Feb 2017, 00:40 Erik Hollensbe, [email protected] wrote:
ye ah I don't think it'd be hard to generate instructions instead of executing them. Box is built to handle stuff like that fairly well.
Would you be willing to do this patch? I can assist with any code reorg required.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/erikh/box/issues/163#issuecomment-281530143, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPWS8G3YYv22UejGpJQ2VrFnlpVzif5ks5re4PqgaJpZM4MGjll .