googleapis
Compute API implementation
What component of google-cloud-cpp is this feature request for?
https://cloud.google.com/compute/docs/reference/rest/v1
Is your feature request related to a problem? Please describe. Would be nice to have this implemented.
Describe the solution you'd like Maybe I'll implement it. I've been doing some work for a client, so depending on their wishes—in terms of time resources—I'll implement the first version for you.
Describe alternatives you've considered FFI with another language implementation.
Additional context How do you usually generate from the discovery JSON? - I'm tempted to write something myself using LLVM's LibClang or LibTooling (probably the latter)…
BTW: There was initially some discussion here a fortnight ago https://github.com/googleapis/google-cloud-cpp/issues/6394
Hi, @SamuelMarks. Thanks for your interest in an API for GCE. As you noticed, that library only has a REST API, so at this point, it's not on our list of APIs to generate using our generator.
The customer demand for a C++ GCE library seems low enough that we don't think a bespoke hand-written library is worth the effort for us to design and maintain. If we were to build ourselves -- or adopt -- a REST-based GCE library we'd be considering points such as:
- Does its API "fit" with the API of our other libraries? I.e., it probably has a
ComputeClientclass that's the main user-facing type. Below that is probably aComputeConnectionlayer w/ virtual functions that's allows customers to mock out the backend for easy testing. Below that is probably some "stub" layer that is not customer facing, but rather it gives us the ability to mock out and test our connection-layer implementations. Without thinking through the details, I of course am not sure that all of this makes sense, but these are the rough layers of our other libraries. - Is it easily maintainable and easily adaptable to new and changing features? I think for GCE, this probably translates to: Is the library autogenerated? I think for something like this, we would not want to maintain a handwritten library. We'd need something that's autogenerated. And then the autogenerator itself would also need to be maintainable and we'd need to discuss its implementation.
- It should have minimal dependencies. Adding external deps is a big pain for customers. So it probably shouldn't rely on more than libcurl, nlohmann::json, and other deps that we already take.
- Is the value to users of having this library worth the cost of maintaining it indefinitely? I have limited headcount for maintaining libraries, and community contributors can come and go at any time. So we need to be careful about what we present as an official Google-provided library vs what's community contributed.
I'm sure there are things I left out, but those are some of the first questions that pop up. Please don't take the above as a checklist for acceptance. Without knowing more it's difficult for me to say whether we would or could accept such generous contribution, own it ourselves, and offer it as an official Google library. However, maybe this could be a community-contributed/supported library? That may be possible.
My advice
It sounds like you have some specific goals with your client. I don't want to block you. My advice would be to design and implement this on your own in your own repo to accomplish your and your clients' goals. Follow our designs/advice as much or as little as you see fit for your goals. When you have something working and nicely taking shape, feel free to let us know. Regardless of whether or not your GCE library ultimately lives in this repo or another, it may still be valuable to others. We could even consider linking to it as a community contributed library. I actually quite like the idea of having somewhere for community contributed libraries. Maybe that's this repo, or another repo. Not sure, but I'll look into this.
Does this align with what you were thinking?
One point I forgot to add: regardless of where your GCE library ends up living, if you have questions, need help, or want advice while implementing it, feel free to ask here and we'll certainly always do our best to help. We still want to help you accomplish your goals with Google Cloud and C++, even if it's not with a library that we directly provide.
Thanks for the design suggestions and offer of support.
In the short-term will just hand-write for a few more endpoints as per my client's wishes. Hopefully medium-term will have the time to write those code-generators, at which point I can offer the contribution back to google-cloud-cpp
[keeping this issue open for others to chime in; and if people want access to my pre-alpha code]
Sure, I would like to see the pre-alpha code. If I learned anything it's that it's important to share code early so that others can chime in and avoid sinking too much time in an approach that ultimately won't work.
PS: Slightly offtopic, but I was finally was able to pitch this to my client's representative. Hoping to close the deal on Tuesday. There are three possibilities:
- [unlikely] My client wants it but won't let me *open-source
- My clients wants it and will let me *open-source
- [likely] My client doesn't want it, so I'll *open-source in my own time (not a high priority but will get done eventually)
*open-source under the (Apache-2.0 OR MIT) license
Ideas for encouraging one:
- Open-source means community will have incentive to—exist and—maintain:
- new features
- documentation
- security fixes [CVE®, for example]
- I will [likely] continue to maintain this after the project has finished, as I'll [likely] be using it for future client projects
Commercially, this means:
- Google will feature this project (e.g., on https://github.com/googleapis/google-cloud-cpp)
- Huge number of potential new use-cases in the multicloud world
- Easy to interface with (because C FTW):
- FFI
- REST API
- RPC APIs
- &etc.
Do you have any suggestions for how I can convince them to go with funding its open-source development?
Thanks
Update: open-source convincing was successful, it's now online at https://github.com/offscale/google-cloud-c
TODO: automatically [re]create the bulk of this library, namely (from the discovery JSON) statically code-generate:
- structs;
- free code
- HTTPS calls
Not sure if the whole pipelining multiple HTTPS calls together will be in or out of scope, but that first TODO is sizeable enough. Confident that I can convince my commercial client for its development: watch this space!
Great, @scotthart want to work together?
Happy to use ROSE or try and tweak LLVM LibTooling for the purposes of code-generating an entire cross-platform—multi crypto-library and multi HTTPS library—library that exposes *all referenced Google Cloud APIs in a straightforward C API.
Alternatively happy to continue on my own [or with others… it's CC0] and use my multi-crypto multi-http-lib abstracting header-only C (C89) libraries and my own C compiler (nonexpanding macro and C89 parser/emitter) to generate the entire library.
*all meaning these: https://www.googleapis.com/discovery/v1/apis/compute/v1/rest
The current plan is to leverage our existing C++ generator to produce a C++ Compute library. We don't intend to author or generate a C version of the library.
Ah I see, you've done it from the Protobuf side. That works I guess. Weird use of enum class for idempotency with those follow-up functions that return different enum values. As for the regular things like getting the IP address from AccessConfig I'd have to look closer through your contributions as rg -i told me where neither nat_ip nor natIP were from d515f8f.
I assume examples are forthcoming, to do basic thinks like:
- Create network
- Create firewall
- Create instance
- Destroy instance
- Destroy firewall
- Destroy network
(it's obvious; albeit verbose [particular on the struct construction]; in my C client library for google-cloud, e.g., https://github.com/offscale/google-cloud-c/blob/9a362f7/google_cloud_c/compute/instance.h#L151)
From a further analysis, I suppose there's AsyncInsertInstances, AsyncAddAccessConfig, AsyncDeleteInstance in your stub https://github.com/googleapis/google-cloud-cpp/blob/d515f8fd/google/cloud/compute/instances/v1/internal/instances_rest_stub.h
(also I'm not sure why these APIs have been generated in such a way as to make the incompatible with C; rather than C/C++ compatible… this means we'll need a new implementation for Rust, Erlang, Swift, Java, Python, Ruby, Go, Zig, D, C#, & others)
Anyway good to see some progress on compute API integration!
You are right about Rust, Erlang, Swift, D, Zig and many other languages. Java, Python, Go, C#, Ruby, NodeJS and PHP have their own libraries:
https://cloud.google.com/compute/docs/api/libraries
I hope we never claimed that the C++ client libraries would be usable to wrap and then use from other languages. That has never been a goal for us.