Request: hard fork and cut releases of the go-bigip library

[dannyk81]

Although not directly related to this repo, but I can't open an issue in the go-bigip repo.

This provider is build on top of https://github.com/f5devcentral/go-bigip, which is a fork of https://github.com/scottdware/go-bigip (which is no longer in active development).

However the fork has seriously diverged over time from the original code.

go-bigip doesn't have any releases and it's not possible to open issues, so it's very tricky to track the changes over time or identify issues introduced by changes that are just pushed to master, it also makes vendoring more difficult.

Proposal:

1. Hard fork https://github.com/f5devcentral/go-bigip and allow to create issues in that repo
2. Cut a release and vendor it in here, future changes should be through a proper release cycle

/cc @scshitole

[dannyk81]

My statement above is inaccurate, @scottdware's repo is indeed in active development (I was thinking about the alternative provider --> https://github.com/DealerDotCom/terraform-provider-bigip), sorry about that.

To be honest, I don't know why go-bigip was forked instead of contributing back to Scott's repo... but the situation now is that this (f5devcentral) fork is diverged way too much and there's no point in keeping the link.

The other points I made are still true regardless :smile:

[scshitole]

We are anyway going a away from the old model and focusing now on AS3 https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/userguide/ Declarative model in that you wont need to focus more on go-bigip most of the stuff will be taken care by JSON payload. The SDK will have only one file with single API.

[dannyk81]

Well.. I don't know what is the roadmap or when all that is going to happen, in the meantime it's rather impossible to work with the go-bigip library as-is...

most of the stuff will be taken care by JSON payload - can you elaborate? what does that mean? go-bigip also generates JSON payloads :smile: SS--> I think this will help https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/userguide/components.html

AS3 seems like a holistic CM solution to manage the BigIP, how is that going to fit into terraform?

Also, AS3 doesn't cover things like:

BUT… it is:

not used to on-board or license a BIG-IP device
not intended for configuring Route Domains, Routes, Self IPs, VLANs, or other Layer 2/3 objects or components
not a mechanism to provide differential authorization or RBAC
not an iApp, nor does it configure iApps
not created to include a graphical interface (GUI)
not a replacement for iWorkflow

So not everything can be covered by AS3 (specifically, this part "not intended for configuring Route Domains, Routes, Self IPs, VLANs, or other Layer 2/3 objects or components")

I'd be interested to see how AS3 will be utilized in a Terraform ecosystem.

The example here just uses null_resource to execute script and push pre-built json files to the BigIP via curl, I'm assuming that this is not the intended end result.

SS--> Example above we are using as the new terraform resource for AS3 is not build yet, once we have the terraform resource for AS3 ready you don't need to use the null resource.

[scshitole]

Here are the details what is covered for BIG-IP on boarding https://clouddocs.f5.com/products/extensions/f5-declarative-onboarding/latest/

[scshitole]

Also AS3 is independent of BIG-IP releases you just need to upload the rpm which handles all the json payload the number of terraform resources required will go down drastically, here is the FAQ link https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/userguide/faq.html

[dannyk81]

Alright, that's all interesting but still doesn't really explain how this is going to work with Terraform.

SS--> We will have something like that for AS3 resource, which will be statefull

// Label is used to identify which Json payload to use
resource "bigip_app_as3"  "as3" {
   label = "Sample 1"
   ident = "someid"
   jsonfile = "${file("sample.json")}"
 }

I understand I can use AS3 to manage BigIP configuration, but the motivation here is slightly different: Terraform is used as a (declarative) framework to define the overall infrastructure state (not just F5) and allow the different related logical components to be declared across multiple systems, e.g. VMs provisioned in VMware using the VMware provider and added to the relevant Pools in F5 using this provider. SS--> This will still remain as it is

Reducing the number of resources doesn't need to be the goal, since the resources are not in a vacuum - they are related to other resources from other parts of the infrastructure (see example for VMs above).

SS--> AS3 will complement what we have lets say you need to deploy GTM for that we dont have terraform resource, now in this case we can use the above AS3 resource and JSON payload to deploy GTM, so it fills the gap for terraform resources. This is just one example there are plenty .

So, if the idea is that we'd need to maintain some JSON file with all the definitions we need for F5 and just have Terraform curl it over - kind of makes the whole thing pretty useless to us.

SS--> Since we don;t have the AS3 resource currently if some one is interested to deploy AS3 null resource helps in filling that gap, but again you don't have to use, this is just intermediate solution.

[dannyk81]

It's a bit confusing when you're editing my post with your comments directly, better to quote reply.

Alright, that's all interesting but still doesn't really explain how this is going to work with Terraform.

SS--> We will have something like that for AS3 resource, which will be statefull

// Label is used to identify which Json payload to use
resource "bigip_app_as3"  "as3" {
   label = "Sample 1"
   ident = "someid"
   jsonfile = "${file("sample.json")}"
 }

this is just a "static" json file that your read and push to bigip, how is that json file content generated? should we use some template data sources? populate it manually?

I understand I can use AS3 to manage BigIP configuration, but the motivation here is slightly different: Terraform is used as a (declarative) framework to define the overall infrastructure state (not just F5) and allow the different related logical components to be declared across multiple systems, e.g. VMs provisioned in VMware using the VMware provider and added to the relevant Pools in F5 using this provider. SS--> This will still remain as it is

sorry, but given the above example - I'm not really clear how that would work.

Reducing the number of resources doesn't need to be the goal, since the resources are not in a vacuum - they are related to other resources from other parts of the infrastructure (see example for VMs above).

SS--> AS3 will complement what we have lets say you need to deploy GTM for that we dont have terraform resource, now in this case we can use the above AS3 resource and JSON payload to deploy GTM, so it fills the gap for terraform resources. This is just one example there are plenty .

So, is it a replacement? or you a side-by-side thing?

So, if the idea is that we'd need to maintain some JSON file with all the definitions we need for F5 and just have Terraform curl it over - kind of makes the whole thing pretty useless to us.

SS--> Since we don;t have the AS3 resource currently if some one is interested to deploy AS3 null resource helps in filling that gap, but again you don't have to use, this is just intermediate solution.

again, given the example you've given above - there's no much difference between using a null_resource and the hypothetical as3 resource, both just take a json file and push it to bigip, my question was about how the content of that json file is to be generated, if the goal is that we create it "manually" or using a template data source - to be honest, I don't find that's very useful or an improvement compared to what the provider gives now.

[scshitole]

It's a bit confusing when you're editing my post with your comments directly, better to quote reply.

Alright, that's all interesting but still doesn't really explain how this is going to work with Terraform. SS--> We will have something like that for AS3 resource, which will be statefull

// Label is used to identify which Json payload to use
resource "bigip_app_as3"  "as3" {
   label = "Sample 1"
   ident = "someid"
   jsonfile = "${file("sample.json")}"
 }

this is just a "static" json file that your read and push to bigip, how is that json file content generated? should we use some template data sources? populate it manually?

Sample Json exists in the documentation, you can take that make changes to the same and use as3 resource to deploy, we have seen DevOps folks like declarative approach ?

I understand I can use AS3 to manage BigIP configuration, but the motivation here is slightly different: Terraform is used as a (declarative) framework to define the overall infrastructure state (not just F5) and allow the different related logical components to be declared across multiple systems, e.g. VMs provisioned in VMware using the VMware provider and added to the relevant Pools in F5 using this provider. SS--> This will still remain as it is

sorry, but given the above example - I'm not really clear how that would work.

Lets say in case of GTM we will have sample GTM existing, you can take that and change as per your requirement, this will be stateful as you will have struct define in the app.go file as shown here https://github.com/terraform-providers/terraform-provider-bigip/blob/dev-branch/vendor/github.com/f5devcentral/go-bigip/app.go all the struct definition will be updated in the same file as we have only single API err := b.post(p, uriMgmt, uriSha, uriAppsvcs, uriDecl)

Reducing the number of resources doesn't need to be the goal, since the resources are not in a vacuum - they are related to other resources from other parts of the infrastructure (see example for VMs above). SS--> AS3 will complement what we have lets say you need to deploy GTM for that we dont have terraform resource, now in this case we can use the above AS3 resource and JSON payload to deploy GTM, so it fills the gap for terraform resources. This is just one example there are plenty .

So, is it a replacement? or you a side-by-side thing?

Its not replacement but fill the gaps for resources not existing

So, if the idea is that we'd need to maintain some JSON file with all the definitions we need for F5 and just have Terraform curl it over - kind of makes the whole thing pretty useless to us. SS--> Since we don;t have the AS3 resource currently if some one is interested to deploy AS3 null resource helps in filling that gap, but again you don't have to use, this is just intermediate solution.

again, given the example you've given above - there's no much difference between using a null_resource and the hypothetical as3 resource, both just take a json file and push it to bigip, my question was about how the content of that json file is to be generated, if the goal is that we create it "manually" or using a template data source - to be honest, I don't find that's very useful or an improvement compared to what the provider gives now.

No there is difference, null resource just dump a static file, when you use as3 resource it will be stateful, that means if you change anything terraform will remember the state information. This is due to the struct definitions in app.go

[dannyk81]

@scshitole can't really follow your answers above as the markdown for the quotes is messed up.

But I feel we are going in circles a bit... the whole point of using Terraform is that the user ("us DevOps folks") has a single declerative interface based on the HCL language, where we define what we want using a standardised approach (i.e. using resources, variables, outputs, data sources, templates, etc..) and how they interact and feed each other information.

Terraform uses the relevant providers to translate these declarations into the proper API.

Sample Json exists in the documentation, you can take that make changes to the same and use as3 resource to deploy,...

^^ Sorry, that makes no sense to me... if I want to write JSON payloads based on some examples and dump them into F5 - I can use some bash with curl or Ansible, I don't follow what is "declarative" about this approach... I can see where this can be used as a gap-stopper for things that the underlying library doesn't support but nothing more.

If I need to author these json files based on some examples/docs, while the content is in most cases derived from output of other resources ?? I don't see how this integrates into terraform at all.

---

I can envision a whole new set of AS3 based resources can be created using the AS3 json payload structs to define the interface in a new library, this would indeed make things easier to maintain over time as it will streamline the underlying library.

Still we will have a appropriate resources/etc defined in HCL so that it can be used in the terraform ecosystem.

That sounds to me like an interesting approach and would indeed make things much easier to maintain and enhance.

---

Having said all that, I would still like a comment on the original proposal of this issue, as I understand go-bigip is not going to go away any time soon (and it will be needed for things not covered by AS3 anyway) so we should have it in a proper repo (where we can open issues) and have regular releases with change tracking.

[dannyk81]

@scshitole would you consider this request?

[scshitole]

@dannyk81 yes we are working on this one.

[jlosito]

Can someone please clarify which project we should be submitting PRs to If we want to contribute to the go rest client? I would like to get some functionality added to this provider but first need to add functionality to the rest client.

[dannyk81]

@jlosito that would be https://github.com/f5devcentral/go-bigip

however this repo is a fork without issue tracking, so indeed a bit of a challenge to work with, hence this request from a while back.

[stobias123]

@scshitole - There's some additional functionality I was looking to add to the provider as well, it's unclear where I need to make those contributions

I'm currently a heavy user of the provider (and prospective contributor), I'll echo the sentiment here that the community needs a clear understanding on the underlying SDK so we can extend and contribute where neccessary.

[KrithikaChidambaram]

Hi, closing this request now. Please re-open if required or send an email to [email protected]. Thanks!