Cannot add instance to Aurora Cluster - (community.aws.rds_instance)

[tonysymbox]

Summary

Trying to add an instance into a previously defined Aurora cluster but get the below error:

"msg": "Unable to create DB instance: An error occurred (InvalidParameterCombination) when calling the CreateDBInstance operation: The requested DB Instance will be a member of a DB Cluster. Set master user password for the DB Cluster."

Issue Type

Bug Report

Component Name

rds_instance

Ansible Version

$ ansible --version

ansible [core 2.11.7] config file = /etc/ansible/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/local/lib/python3.6/site-packages/ansible ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections executable location = /usr/local/bin/ansible python version = 3.6.8 (default, Aug 13 2020, 07:46:32) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] jinja version = 3.0.3 libyaml = True

Collection Versions

$ ansible-galaxy collection list

Collection Version

---

amazon.aws 3.0.0 ansible.posix 1.3.0 community.aws 2.1.0 community.general 4.2.0

AWS SDK versions

$ pip show boto boto3 botocore

Name: boto3 Version: 1.20.27 Summary: The AWS SDK for Python Home-page: https://github.com/boto/boto3 Author: Amazon Web Services Author-email: License: Apache License 2.0 Location: /usr/local/lib/python3.6/site-packages Requires: botocore, jmespath, s3transfer Required-by:

Name: botocore Version: 1.23.29 Summary: Low-level, data-driven core of boto 3. Home-page: https://github.com/boto/botocore Author: Amazon Web Services Author-email: License: Apache License 2.0 Location: /usr/local/lib/python3.6/site-packages Requires: jmespath, python-dateutil, urllib3 Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

OS / Environment

No response

Steps to Reproduce

Playbook:

• name: Create DB Instance community.aws.rds_instance: engine: aurora region: eu-west-1 db_instance_identifier: bla-db instance_type: db.t2.small password: password123 username: admin db_cluster_identifier: bla

Expected Results

Adds instance into cluster

Actual Results

Code of Conduct

• [X] I agree to follow the Ansible Code of Conduct

[markuman]

Try

- name: Create DB Instance
   community.aws.rds_instance:
      engine: aurora
      region: eu-west-1
      db_instance_identifier: bla-db # unique name of a new node
      instance_type: db.t3.small # don't use t2. it's more expensive and slower
      db_cluster_identifier: bla # the name of existing cluster

This is imo a boto3 error message that you see and not a ansible bug.
Maybe username and password should be mutual exclusive with db_cluster_identifier

[tonysymbox]

Try

- name: Create DB Instance
   community.aws.rds_instance:
      engine: aurora
      region: eu-west-1
      db_instance_identifier: bla-db # unique name of a new node
      instance_type: db.t3.small # don't use t2. it's more expensive and slower
      db_cluster_identifier: bla # the name of existing cluster

This is imo a boto3 error message that you see and not a ansible bug. Maybe username and password should be mutual exclusive with db_cluster_identifier

yep that works, thanks Mark!

[markuman]

@jatorcasso do you plan to target also this?

Maybe username and password should be mutual exclusive with db_cluster_identifier

this is not possible atm, because db_cluster_identifier is required.

maybe

• username
• password and
• new_db_cluster_identifier

must be required together and

• username
• password are
• mutual exclusive with db_cluster_identifier.

and it overlaps with the new rds_cluster modules now.
and maybe it results in a breaking change ...

cc @alinabuzachis

[ansibullbot]

cc @jillr @s-hertel @tremble click here for bot help

[jatorcasso]

@markuman I can add this as well as enable the test_aurora test suite and throw some extra tests in there to validate this change

[markuman]

I guess some extra documentation about the difference between rds_instance and rds_cluster would be also helpful as a guidance for users

[alinabuzachis]

@markuman Shouldn't be mutually exclusive instead?

username
password and
new_db_instance_identifier

and of course these ones (as you suggested)

username
password are
mutual exclusive with db_cluster_identifier

Then, this is the actual description for new_db_instance_identifier.

new_db_instance_identifier:
        description:
          - The new DB cluster (lowercase) identifier for the DB cluster when renaming a DB instance. The identifier must contain
            from 1 to 63 letters, numbers, or hyphens and the first character must be a letter and may not end in a hyphen or
            contain consecutive hyphens. Use I(apply_immediately) to rename immediately, otherwise it is updated during the
            next maintenance window.

Shouldn't be - The new DB instance (lowercase) identifier when renaming a DB instance ?

[markuman]

Shouldn't be mutually exclusive instead?

username
password and
new_db_instance_identifier

oh yeah, right.

and of course these ones (as you suggested)

username
password are
mutual exclusive with db_cluster_identifier

Then, this is the actual description for new_db_instance_identifier.

new_db_instance_identifier:
        description:
          - The new DB cluster (lowercase) identifier for the DB cluster when renaming a DB instance. The identifier must contain
            from 1 to 63 letters, numbers, or hyphens and the first character must be a letter and may not end in a hyphen or
            contain consecutive hyphens. Use I(apply_immediately) to rename immediately, otherwise it is updated during the
            next maintenance window.

Shouldn't be - The new DB instance (lowercase) identifier when renaming a DB instance ?

Sounds also good!

[jatorcasso]

@alinabuzachis @markuman we can't make them mutually exclusive since this is only applicable to aurora* instances. We can add a check like

# If engine is aurora, password can't be specified since that's a cluster attribute
is_engine_aurora = False if not module.params.get('engine') else 'aurora' in module.params['engine']
if module.params.get('master_user_password') and is_engine_aurora:
    module.fail_json(msg='It appears you are trying to modify attributes that are managed at the cluster level. Please see rds_cluster.')

but that's not really doing much more than the original error thrown in the first place.

so IMO this is the expected behavior and should be marked as closed

[markuman]

hmmm maybe you're right.
At least the EXAMPLE section should be appended with a replica example. I think that helps people in that case.

In case of RDS (not aurora)

- name: create multi-az mariadb105
  rds_instance:
    db_instance_identifier: mariadb105
    db_parameter_group_name: mdb-div-writer-10-5
    db_instance_class: db.m6g.xlarge
    multi_az: yes
    state: present
    username: "root"
    password: "{{ lookup('onepassword', 'mariadb root password') }}"
    engine: mariadb
    engine_version: 10.5.13
    ...

- name: create read-replica for mariadb105
  rds_instance:
    db_instance_identifier: mariadb105reader
    state: present
    engine: mariadb
    engine_version: 10.5.13
    storage_encrypted: yes
    az: eu-central-1a
    read_replica: yes
    source_db_instance_identifier: mariadb105
    ...

the parameter username and password must be mutually exclusive with read_replica and source_db_instance_identifier.
What do you think @jatorcasso?

[alinabuzachis]

@markuman I guess this is fixed now, right?

[markuman]

@markuman I guess this is fixed now, right?

I think there is nothing to fix....I would call it "optimization" ... that was not made yet.
But because rds_instance has moved to amazon.aws, yes, we can close this issue.