Unable to delete service discovery pool member

[dstokesf5]

Environment

• Application Services Version: 3.28.0
• BIG-IP Version: 16.0.1.1

Summary

If a node from service discovery overlaps with a node configured elsewhere by AS3, then deleting the pool member that discovered that node results in an error.

Steps To Reproduce

Steps to reproduce the behavior:

1. Submit the following declaration. It sets up two things. First is a Consul mock to point service discovery at, the second is an application with a statically defined pool member. The Consul mock will report the same node as the statically defined pool member:

{
    "class": "ADC",
    "schemaVersion": "3.7.0",
    "setup": {
        "class": "Tenant",
        "consulMock": {
            "class": "Application",
            "vip": {
                "class": "Service_HTTP",
                "virtualAddresses": [
                    "192.0.2.0"
                ],
                "virtualPort": 8082,
                "iRules": [
                    {
                        "use": "consul_mock_irule"
                    }
                ]
            },
            "consul_mock_irule": {
                "class": "iRule",
                "iRule": {
                    "base64": "d2hlbiBIVFRQX1JFUVVFU1QgewogICAgc2V0IHJlc3BvbnNlIHtbewoJIklEIjogIjQ0RDJFOTk3LTI0NjItNDEwMy1COEYwLTQyOUUwRDgzMTcyRiIsCgkiTm9kZSI6ICIxOTIuMC4yLjEiLAoJIkFkZHJlc3MiOiAiMTkyLjAuMi4xIgp9XQp9CiAgICBIVFRQOjpyZXNwb25kIDIwMCBjb250ZW50ICIkcmVzcG9uc2UiIENvbnRlbnQtdHlwZSAiYXBwbGljYXRpb24vanNvbiIKfQ=="
                }
            }
        },
        "conflict": {
            "class": "Application",
            "pool": {
                "class": "Pool",
                "members": [
                    {
                        "shareNodes": true,
                        "serverAddresses": [
                            "192.0.2.1"
                        ],
                        "servicePort": 30036
                    }
                ]
            }
        }
    }
}

2. Submit a second declaration that makes use of the Consul mock:

{
    "class": "ADC",
    "schemaVersion": "3.7.0",
    "controls": {
        "traceResponse": true
    },
    "tenant": {
        "class": "Tenant",
        "application": {
            "class": "Application",
            "vip": {
                "class": "Service_HTTP",
                "virtualAddresses": [
                    "192.0.2.3"
                ],
                "virtualPort": 443,
                "pool": "pool"
            },
            "pool": {
                "class": "Pool",
                "members": [
                    {
                        "servicePort": 30033,
                        "addressDiscovery": "consul",
                        "autoPopulate": true,
                        "uri": "http://192.0.2.0:8082/catalog/nodes",
                        "updateInterval": 5,
                        "rejectUnauthorized": false,
                        "shareNodes": true
                    }
                ]
            }
        }
    }
}

3. Remove the service discovery pool member:

{
    "class": "ADC",
    "schemaVersion": "3.7.0",
    "controls": {
        "traceResponse": true
    },
    "tenant": {
        "class": "Tenant",
        "application": {
            "class": "Application",
            "vip": {
                "class": "Service_HTTP",
                "virtualAddresses": [
                    "192.0.2.3"
                ],
                "virtualPort": 443,
                "pool": "pool"
            },
            "pool": {
                "class": "Pool",
                "members": [
                ]
            }
        }
    }
}

4. Observer the following error response:

    "results": [
        {
            "code": 422,
            "message": "declaration failed",
            "response": "01020036:3: The requested Pool Member (/tenant/application/pool /Common/192.0.2.1 30033) was not found.",
            "host": "localhost",
            "tenant": "tenant",
            "runTime": 6688
        }
    ],

Expected Behavior

The service discovery pool member should be deleted with no errors, and the node should remain because it is still in use by AS3.

Actual Behavior

It looks like AS3 might be having a problem with reference counting and is trying to delete a node that should not be deleted when the service discovery pool member is removed.

Notes

This was reported in a comment of #461.

[dstokesf5]

Added to our internal product backlog as AUTOTOOL-2556.