f5-ansible icon indicating copy to clipboard operation
f5-ansible copied to clipboard

bigip_configsync_action with overwrite_config: yes can return "Recommended action: Synchronize to group

Open jmcguir opened this issue 1 year ago • 12 comments

COMPONENT NAME bigip_configsync_action

Environment ANSIBLE VERSION

ansible-playbook 2.9.11
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]

BIGIP VERSION

bigip:15.1.10.2

OS / ENVIRONMENT

'f5networks.f5_modules:1.26.0'

SUMMARY Checking https://github.com/ansible/ansible_collections_f5/blob/master/plugins/modules/bigip_configsync_action.py

I would expect that bigip_configsync_action with overwrite_config: yes would be equivalent to force-full-load-push so when running the following code:

  • name: Perform Sync bigip_configsync_action: device_group: "F5-LTM-PAIR1" sync_device_to_group: yes overwrite_config: yes timeout: 60

I would never get the following message:

{
    "msg": "Recommended action: Synchronize this device to group F5-LTM-PAIR-GROUP",
    "invocation": {
        "module_args": {
            "device_group": "F5-LTM-PAIR-GROUP",
            "sync_device_to_group": true,
            "overwrite_config": true,
            "provider": {
                "server": "F5-LTM-PAIR1",
                "user": "USERNAME",
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "validate_certs": false,
                "server_port": 443,
                "transport": "rest",
                "timeout": null,
                "no_f5_teem": false,
                "auth_provider": null
            },
            "sync_group_to_device": null
        }
    },
    "_ansible_no_log": false,
    "changed": false
}

How is that this can happen? How this could be made more reliable?

STEPS TO REPRODUCE Configure two units with basic config (time sync'ed) Configure services in one of the units. Save the config Perform sync to the device-group as above

EXPECTED RESULTS The cluster is always in sync when using this option. The condition "Recommended action: Synchronize xxx to group lb-cluster" is not handled by _wait_for_sync

I tried to reopen https://github.com/F5Networks/f5-ansible/issues/2065 but I couldn't. This seems to be the same issue. I don't understand why it's closed.

jmcguir avatar Oct 27 '23 16:10 jmcguir

Hi,

Thanks for reporting. Added to the backlog and internal tracking ID for this request is: INFRAANO-433.

pgouband avatar Nov 16 '23 08:11 pgouband

Hello @jmcguir , I ran the task you shared on bigip 15, and it worked fine in my case. I didn't get the "Recommended action: Synchronize this device to group F5-LTM-PAIR-GROUP" message.

urohit011 avatar Dec 04 '23 04:12 urohit011

Hey @urohit011, and you made a change on the standby F5 and then tried to sync from the standby to the device group?

jmcguir avatar Dec 04 '23 16:12 jmcguir

Hi @jmcguir , I tried from the standby device and again it worked fine without issues.

urohit011 avatar Dec 06 '23 05:12 urohit011

Hi @jmcguir,

We think it will be more efficient to open a case via https://my.f5.com/ so support can check if the sync issue is related to the BIG-IP config and we can ask you to provide a qkview, logs and ucs.

pgouband avatar Dec 07 '23 17:12 pgouband

@urohit011 and @pgouband you are following the steps to reproduce including saving the config?

I can consistently make this happen.

I believe this function is the issue: def _validate_pending_status(self, details) https://github.com/F5Networks/f5-ansible/blob/68124ba2bff5fa20c6f383821b3a61756bea2f0e/ansible_collections/f5networks/f5_modules/plugins/modules/bigip_configsync_action.py#L365C13-L365C13

I mean it already says its a hack in the comment the line below :).

Because when I curl https:///f5-alb01.test.com/mgmt/tm/cm/sync-status

I get back this:

"kind": "tm:cm:sync-status:sync-statusstats",
"selfLink": "https://localhost/mgmt/tm/cm/sync-status?ver=17.1.1",
"entries": {
    "https://localhost/mgmt/tm/cm/sync-status/0": {
        "nestedStats": {
            "entries": {
                "color": {
                    "description": "red"
                },
                "https://localhost/mgmt/tm/cm/syncStatus/0/details": {
                    "nestedStats": {
                        "entries": {
                            "https://localhost/mgmt/tm/cm/syncStatus/0/details/0": {
                                "nestedStats": {
                                    "entries": {
                                        "details": {
                                            "description": "f5name.com: connected"
                                        }
                                    }
                                }
                            },
                            "https://localhost/mgmt/tm/cm/syncStatus/0/details/1": {
                                "nestedStats": {
                                    "entries": {
                                        "details": {
                                            **"description": "F5-DEVICE-GROUP (Changes Pending): There is a possible change conflict between f5-alb01.test.com and f5-alb02.test.com."**
                                        }
                                    }
                                }
                            },
                            "https://localhost/mgmt/tm/cm/syncStatus/0/details/2": {
                                "nestedStats": {
                                    "entries": {
                                        "details": {
                                            **"description": " - Recommended action: Synchronize f5-alb01.test.com to group F5-DEVICE-GROUP"**
                                        }
                                    }
                                }
                            }
                        }
                    }
                },
                "mode": {
                    "description": "high-availability"
                },
                "status": {
                    "description": "Changes Pending"
                },
                "summary": {
                    "description": "There is a possible change conflict between f5-alb01.test.com. and f5-alb02.test.com."
                }
            }
        }
    }
}

}

You can see that "description": " - Recommended action: Synchronize f5-alb01.test.com to group F5-DEVICE-GROUP" is present where validate_pending_status is looking for it. This having the effect of causing the job to error out. For reference I'm running the sync on the active (f5-alb02).

The fix would be to either wait for a bit longer for the device to sync before polling or to handle that if we pass overwrite that we don't care about this particular error: Recommended action: Synchronize f5-alb01.test.com to group F5-DEVICE-GROUP". Since we are obviously overwriting. I mean think about it logically. Does that recommended action make any sense in comparison to what I'm trying to do? I'm telling the f5 to ignore anything and overwrite the config.

I don't see how a config item would be effecting this. Please look at https://github.com/F5Networks/f5-ansible/issues/2065 for further evidence of this.

jmcguir avatar Dec 19 '23 20:12 jmcguir

Hi @jmcguir , I followed all the steps you mentioned but I didn't see the error. But, about saving the config could you tell me how do you go about saving the config I guess it happens automatically in the GUI, or are you doing it differently, or am I missing something?

urohit011 avatar Jan 12 '24 12:01 urohit011

Hey @urohit011 I'm saving the config with ansible

  • name: SAVE RUNNING CONFIG ON BIG-IP bigip_config: provider: "{{provider}}" save: yes

The key being after the save on one side there is a config difference making the pair out of sync.

jmcguir avatar Jan 30 '24 19:01 jmcguir

Could you please provide the entire playbook you're running, @jmcguir? Thanks

urohit011 avatar Jan 31 '24 05:01 urohit011

I can't share the whole playbook publicly as it has employer specific details. It's 1000's of lines long and is made up of python and ansible. I've opened case 00550506 @urohit011

jmcguir avatar Jan 31 '24 14:01 jmcguir

Hi @jmcguir,

We are able to access to the file you uploaded but we think it's better to open a case and to ask to escalate it to Ansible dev team so we can discuss via support as we may need some information. I apologize if there was a misunderstanding about the case. Could you open a new case and ask to escalate it to Ansible dev team?

pgouband avatar Feb 13 '24 16:02 pgouband

Hi @pgouband I've reopened the case and escalated it to the ansible dev team.

jmcguir avatar Feb 13 '24 19:02 jmcguir

Hi @jmcguir , I tried running the playbook without using the vault protected var files and the roles and the configsync_action task ran fine without any issue.

urohit011 avatar Mar 11 '24 10:03 urohit011

Okay now uncomment lines 363 - 368 in upgrade_bigip.yml and report back to me please.

Did you look at this https://github.com/F5Networks/f5-ansible/issues/2378#issuecomment-1863439650?

I linked some source code that should inform the issue.

jmcguir avatar Mar 11 '24 18:03 jmcguir

@jmcguir Working on it

urohit011 avatar Mar 12 '24 11:03 urohit011

Hi,

We tried to reproduce the issue in our lab. Also support team tried too without success and there was no response from you. The issue is maybe in your environment. Please reopen a support case if the issue is still occurring.

pgouband avatar Apr 10 '24 08:04 pgouband