grailbio
Backend for GCP
Took a very preliminary look at this today.
I'm likely to become distracted by my other projects quickly but I'd like to contribute something to this, even if only guideposts.
The code creates GCE VMs but that's about it. Once I get a feel for it, I'll refactor the code to cleanly separate bigmachine and Google functionality.
I'm struggling because I've not yet got a working local version to use as a guide (#4).
The flags mechanism in driver.go won't easily scale to supporting multiple backends and will need some work.
Hopefully providing a second implementation will help generalize the code too.
I continue to think that Kubernetes (#3) is a good backend model to consider.
Very cool!
About the flag mechanism: yes, this is known. This is one of the reasons we use the config package in Bigslice. Note also that using the driver package is entirely optional; it's easy enough to configure and use Bigmachine in straightforward ways.
Bigmachine already has support for the aforementioned config package; so you can already (with the right imports), do things like:
var b *bigmachine.B
config.Must("bigmachine", &b)
to create bigmachine sessions using the user's configuration.
Bigslice has a tool that is intended to help the user configure the cloud provider (in AWS, this reduces to just creating a security group with the right network ACLs).
This is at a point where it's functional:
https://github.com/DazWilkin/bigmachine/tree/compute-engine