ansible-community
molecule[ec2] documentation
Building on #266, I would love the same.
Finding /molecule-plugins/test/ec2/scenarios/driver/ec2/molecule/default/molecule.yml I'm like woot, woot, until I realize the ../../../../../resources/playbooks/ec2/create.yml and ../../../../../resources/playbooks/ec2/destroy.yml playbooks aren't included in the repo.
https://github.com/ansible-community/molecule-plugins/tree/9943538d487b25a03d24c886980f3d0a643aaaaa/src/molecule_plugins/ec2/cookiecutter/%7B%7Bcookiecutter.molecule_directory%7D%7D/%7B%7Bcookiecutter.scenario_name%7D%7D
and molecule init scenario -d ec2 myrepo should create it for you.
[...]
TASK [Expand templates] ********************************************************
changed: [localhost] => (item=molecule/myrepo/molecule.yml)
changed: [localhost] => (item=molecule/myrepo/destroy.yml)
changed: [localhost] => (item=molecule/myrepo/create.yml)
changed: [localhost] => (item=molecule/myrepo/converge.yml)
Yeah, but these are partially populated stubs. I assume from this and the lack of docs I am to write ansible playbooks referencing keys declared in the molecule file.
no, just write your standard ansible roles, generate the molecule skeleton, update the molecule.yml with platforms and such. after that you should add tests to verify.yml and then run molecule test.
in https://github.com/konstruktoid/ansible-role-template/tree/main/molecule, I got the default scenario that uses vagrant and an additional one using podman.
I did this to start... the ec2 isn't created... let me loop back and simplify my molecule.yml to the bare minimum and try that again. The create.yml and destroy.yml created by the init using the ec2 driver will need to be deleted based in what you have said but these files existing stubbed out after the init had me thinking otherwise... the lack of docs didn't give anything to gut check against vice reading the code.
I'm not defending the lack of documentation in any way, but I've never had to create or modifiy create.yml or destroy.yml
fwiw, for some plugins (I guess it's including ec2), there's an issue coming from molecule. Cookiecutter support has been removed (and I don't remember seeing some kind of replacement possible from -plugins point of view). The leads to the issue of molecule init bringing "useless" molecule file like create.yml and destroy.yml files intended (for delegated driver iirc?). Most of times, the solution is just to remove these two files. People using podman/docker plugins are not seeing that as molecule has been patched to not copy these files when they're used.
When using the EC2 plugin?
Okay. This good to know. The fault is likely with my molecule.yml and the stubbed out create.yml and destroy.yml being created by the init. I will revisit this morning.
Hmm.
I'm on Ubuntu 22.04.4 LTS using pyenv and poetry to install molecule and its dependencies:
➜ molecule --version
molecule 24.8.0 using python 3.10
ansible:2.16.10
azure:23.5.3 from molecule_plugins
containers:23.5.3 from molecule_plugins requiring collections: ansible.posix>=1.3.0 community.docker>=1.9.1 containers.podman>=1.8.1
default:24.8.0 from molecule
docker:23.5.3 from molecule_plugins requiring collections: community.docker>=3.4.11 ansible.posix>=1.4.0
ec2:23.5.3 from molecule_plugins
gce:23.5.3 from molecule_plugins requiring collections: google.cloud>=1.0.2 community.crypto>=1.8.0
openstack:23.5.3 from molecule_plugins requiring collections: openstack.cloud>=2.1.0
podman:23.5.3 from molecule_plugins requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
vagrant:23.5.3 from molecule_plugins
My Python dependency tree is
molecule-plugins 23.5.3 Molecule Plugins
└── molecule >=6.0.0a1
├── ansible-compat >=24.6.1
│ ├── ansible-core >=2.14
│ │ ├── cryptography *
│ │ │ └── cffi >=1.12
│ │ │ └── pycparser *
│ │ ├── jinja2 >=3.0.0
│ │ │ └── markupsafe >=2.0
│ │ ├── packaging *
│ │ ├── pyyaml >=5.1
│ │ └── resolvelib >=0.5.3,<1.1.0
│ ├── jsonschema >=4.6.0
│ │ ├── attrs >=22.2.0
│ │ ├── jsonschema-specifications >=2023.03.6
│ │ │ └── referencing >=0.31.0
│ │ │ ├── attrs >=22.2.0 (circular dependency aborted here)
│ │ │ └── rpds-py >=0.7.0
│ │ ├── referencing >=0.28.4 (circular dependency aborted here)
│ │ └── rpds-py >=0.7.1 (circular dependency aborted here)
│ ├── packaging * (circular dependency aborted here)
│ ├── pyyaml * (circular dependency aborted here)
│ └── subprocess-tee >=0.4.1
├── ansible-core >=2.12.10 (circular dependency aborted here)
├── click >=8.0,<9
│ └── colorama *
├── click-help-colors *
│ └── click >=7.0,<9 (circular dependency aborted here)
├── enrich >=1.2.7
│ └── rich >=9.5.1
│ ├── markdown-it-py >=2.2.0
│ │ └── mdurl >=0.1,<1.0
│ └── pygments >=2.13.0,<3.0.0
├── jinja2 >=2.11.3 (circular dependency aborted here)
├── jsonschema >=4.9.1 (circular dependency aborted here)
├── packaging * (circular dependency aborted here)
├── pluggy >=0.7.1,<2.0
├── pyyaml >=5.1 (circular dependency aborted here)
├── rich >=9.5.1 (circular dependency aborted here)
└── wcmatch >=8.1.2
└── bracex >=2.1.1
I'm authenticated into the aws cli and verified I can pull the contents from s3 via aws s3 ls.
My molecule folder only holds {{project path}}/molecule/grr/molecule.yml and nothing else
The contents for {{project path}}/molecule/grr/molecule.yml are (some of which I had to guess at referring to the defaults https://github.com/ansible-community/molecule-plugins/blob/main/src/molecule_plugins/ec2/cookiecutter/%7B%7Bcookiecutter.molecule_directory%7D%7D/%7B%7Bcookiecutter.scenario_name%7D%7D/create.yml#L19) -- I later found and verified against https://github.com/ansible-community/molecule-plugins/blob/main/doc/ec2/platforms.rst:
---
driver:
name: ec2
platforms:
- name: instance
image: ami-0ce8fc041db68907c
instance_type: t2.micro
vpc_subnet_id: subnet-06543a923334a6b41
security_groups::
- sg-0f93b0e3a1d7626dc
- sg-0924b78790d0d9590
- sg-0bb3f716a64b69244
- sg-0737a1bcac58b9a2d
region: us-gov-west-1
assign_public_ip: false
key_name: MY_RSA
private_key_path: ~/.ssh/MY_RSA
ssh_user: ec2-user
provisioner:
name: ansible
env:
ANSIBLE_COLLECTIONS_PATH: ../../ansible_collections
playbooks:
converge: ../../linux.yml
verifier:
name: ansible
Executing molecule --debug -v create --scenario-name grr --driver-name ec2 doesn't create the instance
DEBUG Validating schema /home/nemonik/Development/workspace/playbook_prometheus_node_exporter/molecule/grr/molecule.yml.
WARNING Driver ec2 does not provide a schema.
INFO grr scenario test matrix: dependency, create, prepare
INFO Performing prerun with role_name_check=0...
INFO Running grr > dependency
WARNING Skipping, missing the requirements file.
WARNING Skipping, missing the requirements file.
INFO Running grr > create
WARNING Skipping, instances already created.
INFO Running grr > prepare
WARNING Skipping, prepare playbook not configured.
Well, it sez it does
➜ molecule list
WARNING Driver ec2 does not provide a schema.
WARNING Driver ec2 does not provide a schema.
WARNING Driver ec2 does not provide a schema.
INFO Running grr > list
╷ ╷ ╷ ╷ ╷
Instance Name │ Driver Name │ Provisioner Name │ Scenario Name │ Created │ Converged
╶──────────────────────────────────────────┼─────────────┼──────────────────┼───────────────┼─────────┼───────────╴
molecule_grr │ ec2 │ ansible │ grr │ true │ false
But it doesn't as there no instance to ansible:
➜ molecule --debug -v converge --scenario-name grr
DEBUG Validating schema /home/nemonik/Development/workspace/playbook_prometheus_node_exporter/molecule/grr/molecule.yml.
WARNING Driver ec2 does not provide a schema.
INFO grr scenario test matrix: dependency, create, prepare, converge
INFO Performing prerun with role_name_check=0...
INFO Running grr > dependency
WARNING Skipping, missing the requirements file.
WARNING Skipping, missing the requirements file.
INFO Running grr > create
WARNING Skipping, instances already created.
INFO Running grr > prepare
WARNING Skipping, prepare playbook not configured.
INFO Running grr > converge
DEBUG: ANSIBLE ENVIRONMENT:
ANSIBLE_COLLECTIONS_PATH: ../../ansible_collections
ANSIBLE_CONFIG: /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/ansible.cfg
ANSIBLE_FILTER_PLUGINS: /home/nemonik/.cache/pypoetry/virtualenvs/playbook-prometheus-node-exporter-TYBnf8te-py3.10/lib/python3.10/site-packages/molecule/provisioner/ansible/plugins/filter:/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/plugins/filter:/home/nemonik/Development/workspace/playbook_prometheus_node_exporter/plugins/filter:/home/nemonik/.ansible/plugins/filter:/usr/share/ansible/plugins/filter
ANSIBLE_FORCE_COLOR: '1'
ANSIBLE_LIBRARY: /home/nemonik/.cache/pypoetry/virtualenvs/playbook-prometheus-node-exporter-TYBnf8te-py3.10/lib/python3.10/site-packages/molecule/provisioner/ansible/plugins/modules:/home/nemonik/.cache/pypoetry/virtualenvs/playbook-prometheus-node-exporter-TYBnf8te-py3.10/lib/python3.10/site-packages/molecule_plugins/vagrant/modules:/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/library:/home/nemonik/Development/workspace/playbook_prometheus_node_exporter/library:/home/nemonik/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
ANSIBLE_ROLES_PATH: /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/roles:/home/nemonik/Development/workspace:/home/nemonik/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
ANSIBLE_VERBOSITY: '1'
DEBUG: MOLECULE ENVIRONMENT:
MOLECULE_DEBUG: 'True'
MOLECULE_DEPENDENCY_NAME: galaxy
MOLECULE_DRIVER_NAME: ec2
MOLECULE_ENV_FILE: /home/nemonik/Development/workspace/playbook_prometheus_node_exporter/.env.yml
MOLECULE_EPHEMERAL_DIRECTORY: /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr
MOLECULE_FILE: /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/molecule.yml
MOLECULE_INSTANCE_CONFIG: /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/instance_config.yml
MOLECULE_INVENTORY_FILE: /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/inventory/ansible_inventory.yml
MOLECULE_PROJECT_DIRECTORY: /home/nemonik/Development/workspace/playbook_prometheus_node_exporter
MOLECULE_PROVISIONER_NAME: ansible
MOLECULE_SCENARIO_DIRECTORY: /home/nemonik/Development/workspace/playbook_prometheus_node_exporter/molecule/grr
MOLECULE_SCENARIO_NAME: grr
MOLECULE_STATE_FILE: /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/state.yml
MOLECULE_VERIFIER_NAME: ansible
MOLECULE_VERIFIER_TEST_DIRECTORY: /home/nemonik/Development/workspace/playbook_prometheus_node_exporter/molecule/grr/tests
DEBUG: SHELL REPLAY:
ANSIBLE_COLLECTIONS_PATH=../../ansible_collections ANSIBLE_CONFIG=/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/ansible.cfg ANSIBLE_FILTER_PLUGINS=/home/nemonik/.cache/pypoetry/virtualenvs/playbook-prometheus-node-exporter-TYBnf8te-py3.10/lib/python3.10/site-packages/molecule/provisioner/ansible/plugins/filter:/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/plugins/filter:/home/nemonik/Development/workspace/playbook_prometheus_node_exporter/plugins/filter:/home/nemonik/.ansible/plugins/filter:/usr/share/ansible/plugins/filter ANSIBLE_FORCE_COLOR=1 ANSIBLE_LIBRARY=/home/nemonik/.cache/pypoetry/virtualenvs/playbook-prometheus-node-exporter-TYBnf8te-py3.10/lib/python3.10/site-packages/molecule/provisioner/ansible/plugins/modules:/home/nemonik/.cache/pypoetry/virtualenvs/playbook-prometheus-node-exporter-TYBnf8te-py3.10/lib/python3.10/site-packages/molecule_plugins/vagrant/modules:/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/library:/home/nemonik/Development/workspace/playbook_prometheus_node_exporter/library:/home/nemonik/.ansible/plugins/modules:/usr/share/ansible/plugins/modules ANSIBLE_ROLES_PATH=/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/roles:/home/nemonik/Development/workspace:/home/nemonik/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles ANSIBLE_VERBOSITY=1 MOLECULE_DEBUG=True MOLECULE_DEPENDENCY_NAME=galaxy MOLECULE_DRIVER_NAME=ec2 MOLECULE_ENV_FILE=/home/nemonik/Development/workspace/playbook_prometheus_node_exporter/.env.yml MOLECULE_EPHEMERAL_DIRECTORY=/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr MOLECULE_FILE=/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/molecule.yml MOLECULE_INSTANCE_CONFIG=/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/instance_config.yml MOLECULE_INVENTORY_FILE=/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/inventory/ansible_inventory.yml MOLECULE_PROJECT_DIRECTORY=/home/nemonik/Development/workspace/playbook_prometheus_node_exporter MOLECULE_PROVISIONER_NAME=ansible MOLECULE_SCENARIO_DIRECTORY=/home/nemonik/Development/workspace/playbook_prometheus_node_exporter/molecule/grr MOLECULE_SCENARIO_NAME=grr MOLECULE_STATE_FILE=/home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/state.yml MOLECULE_VERIFIER_NAME=ansible MOLECULE_VERIFIER_TEST_DIRECTORY=/home/nemonik/Development/workspace/playbook_prometheus_node_exporter/molecule/grr/tests
Using /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/ansible.cfg as config file
PLAY [all] *********************************************************************
TASK [Gathering Facts] *********************************************************
fatal: [molecule_grr]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname molecule_grr: Temporary failure in name resolution", "unreachable": true}
PLAY RECAP *********************************************************************
molecule_grr : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0
CRITICAL Ansible return code was 4, command was: ansible-playbook --diff --inventory /home/nemonik/.cache/molecule/playbook_prometheus_node_exporter/grr/inventory --skip-tags molecule-notest,notest /home/nemonik/Development/workspace/playbook_prometheus_node_exporter/linux.yml
Adding back in the create.yml and destroy.yml created by molecule init configured to use the ec2 drive have no impact when rerunning the create.
My organization requires MFA, so I am setting these environment variables:
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_SESSION_TOKEN
- AWS_CA_BUNDLE
- AWS_PROFILE
@ssbarnea thoughts?
To get the create to work
I had to go to v23.5.3's
https://github.com/ansible-community/molecule-plugins/blob/v23.5.3/src/molecule_plugins/ec2/cookiecutter/%7B%7Bcookiecutter.molecule_directory%7D%7D/%7B%7Bcookiecutter.scenario_name%7D%7D/create.yml
and copy the content from between {% raw -%} and {%- endraw %} lines into my scenerio's create.yml... I suspect I'll need to do that for destroy.yml too...
And yep... do the same with destory.yml... and molecule works.
I guess this is cuz cookiecutter support was removed or something?
I can confirm that it's reproduced in:
molecule --version
INFO Found config file /ansible/.config/molecule/config.yml
molecule 25.3.1 using python 3.11
ansible:2.18.3
podman:23.7.0 from molecule_plugins requiring collections: containers.podman>=1.7.0 ansible.posix>=1.3.0
openstack:23.7.0 from molecule_plugins requiring collections: openstack.cloud>=2.1.0
gce:23.7.0 from molecule_plugins requiring collections: google.cloud>=1.0.2 community.crypto>=1.8.0
default:25.3.1 from molecule
vagrant:23.7.0 from molecule_plugins
docker:23.7.0 from molecule_plugins requiring collections: community.docker>=3.10.2 ansible.posix>=1.4.0
containers:23.7.0 from molecule_plugins requiring collections: ansible.posix>=1.3.0 community.docker>=1.9.1 containers.podman>=1.8.1
azure:23.7.0 from molecule_plugins
ec2:23.7.0 from molecule_plugins
Meaning that ec2 plugin doesn't work out of the box and without the workaround that @nemonik found.
Can it be fixed?
