community.aws icon indicating copy to clipboard operation
community.aws copied to clipboard

aws_ssm connection: add support for SSM document

Open blinkseb opened this issue 3 years ago • 17 comments

SUMMARY

This PR adds support for SSM document to the SSM connection plugin.

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

community.aws.aws_ssm

ADDITIONAL INFORMATION

The new document parameters is directly forwarded to the SSM start_session method.

Usage:

- name: Install a Nginx Package
  vars:
    ansible_connection: aws_ssm
    ansible_aws_ssm_bucket_name: nameofthebucket
    ansible_aws_ssm_region: us-west-2
    ansible_aws_ssm_document: nameofthecustomdocument
  tasks:
    - name: Install a Nginx Package
      yum:
        name: nginx
        state: present

blinkseb avatar Jan 19 '22 08:01 blinkseb

Thanks for your review! I added the changelog fragment and tried something for the integration tests

blinkseb avatar Jan 26 '22 07:01 blinkseb

@blinkseb It looks like the integration tests are currently disabled: https://github.com/ansible-collections/community.aws/blob/main/tests/integration/targets/connection_aws_ssm/aliases

I'm not sure of the history of the past issues and if they are resolved or not etc, but you can try running them locally to verify all is working as expected you would need to specify the --allow-disabled flag so for example:

ansible-test integration --docker default -vvv connection_aws_ssm --allow-disabled

marknet15 avatar Feb 04 '22 17:02 marknet15

ok I'm progressing. I re-enabled the integration tests, like it was done in https://github.com/ansible-collections/community.aws/pull/763. I'm currently stuck with a permission error to create a new SSM document, so I opened a PR to add the necessary permissions here: https://github.com/mattclay/aws-terminator/pull/192

blinkseb avatar Feb 07 '22 14:02 blinkseb

I run the tests locally. I guess it all passes.
But at the end, the test still return exit code 1. No idea why.

....
localhost                  : ok=16   changed=6    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   

AWS ACTIONS: ['ec2:DescribeInstances', 'ec2:TerminateInstances', 'iam:DeleteRole', 'iam:DetachRolePolicy', 'iam:GetRole', 'iam:ListAttachedRolePolicies', 'iam:ListInstanceProfilesForRole', 'iam:RemoveRoleFromInstanceProfile', 'kms:DescribeKey', 'kms:GetKeyPolicy', 'kms:GetKeyRotationStatus', 'kms:ListAliases', 'kms:ListGrants', 'kms:ListKeyPolicies', 'kms:ListResourceTags', 'kms:ScheduleKeyDeletion']
Command exited with status 1 after 567.5553092956543 seconds.
Run command: /usr/bin/python /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.01774883270263672 seconds.
Run command: /usr/bin/python2 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python2
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.01106405258178711 seconds.
Run command: /usr/bin/python2.6 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python2.6
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.23974013328552246 seconds.
Run command: /usr/bin/python2.7 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python2.7
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.01092529296875 seconds.
Run command: /usr/bin/python3 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python3
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.016592025756835938 seconds.
Run command: /usr/bin/python3.5 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python3.5
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.0272824764251709 seconds.
Run command: /usr/bin/python3.6 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python3.6
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.016504526138305664 seconds.
Run command: /usr/bin/python3.7 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python3.7
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.014443159103393555 seconds.
Run command: /usr/bin/python3.8 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python3.8
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.01379251480102539 seconds.
Run command: /usr/bin/python3.9 /root/ansible/test/lib/ansible_test/_data/versions.py
Working directory: /root/ansible_collections/community/aws
Program found: /usr/bin/python3.9
HOME=/root
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Command exited with status 0 after 0.015276432037353516 seconds.
NOTICE: To resume at this test target, use the option: --start-at connection_aws_ssm
ERROR: Command "./runme.sh -vvvv -e '@/root/ansible_collections/community/aws/tests/output/.tmp/integration/connection_aws_ssm-9b3389a1-ÅÑŚÌβŁÈ/tests/integration/config-file-tpixtwm6.json'" returned exit status 1.
Cleaning up temporary python directory: /tmp/python-73fse3in-ansible
Command exited with status 1 after 608.0626201629639 seconds.
Run command: docker exec 69a0350a43534dd4965c88f01d5c86423716ab768a1e32ebf9f5dcae60ce5fba tar czf /root/output.tgz --exclude .tmp -C /root/ansible_collections/community/aws/tests output
Working directory: /home/m/git/ansible_collections/community/aws
Program found: /usr/bin/docker
HOME=/home/m
LC_ALL=en_US.UTF-8
PATH=/home/m/.local/bin:/usr/local/bin:/usr/bin:/var/lib/snapd/snap/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/m/.local/bin/
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
Command exited with status 0 after 0.08580565452575684 seconds.
Run command: docker exec -i 69a0350a43534dd4965c88f01d5c86423716ab768a1e32ebf9f5dcae60ce5fba dd if=/root/output.tgz bs=65536
Working directory: /home/m/git/ansible_collections/community/aws
Program found: /usr/bin/docker
HOME=/home/m
LC_ALL=en_US.UTF-8
PATH=/home/m/.local/bin:/usr/local/bin:/usr/bin:/var/lib/snapd/snap/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/m/.local/bin/
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
Command exited with status 0 after 0.08487987518310547 seconds.
Run command: tar oxzf /tmp/ansible-result-h_w2h_mw.tgz -C /home/m/git/ansible_collections/community/aws/tests
Working directory: /home/m/git/ansible_collections/community/aws
Program found: /usr/bin/tar
HOME=/home/m
LC_ALL=en_US.UTF-8
PATH=/home/m/.local/bin:/usr/local/bin:/usr/bin:/var/lib/snapd/snap/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/m/.local/bin/
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
Command exited with status 0 after 0.006506204605102539 seconds.
Run command: docker rm -f 69a0350a43534dd4965c88f01d5c86423716ab768a1e32ebf9f5dcae60ce5fba
Working directory: /home/m/git/ansible_collections/community/aws
Program found: /usr/bin/docker
HOME=/home/m
LC_ALL=en_US.UTF-8
PATH=/home/m/.local/bin:/usr/local/bin:/usr/bin:/var/lib/snapd/snap/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/m/.local/bin/
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
Command exited with status 0 after 0.5264806747436523 seconds.
ERROR: Command "docker exec 69a0350a43534dd4965c88f01d5c86423716ab768a1e32ebf9f5dcae60ce5fba /usr/bin/env ANSIBLE_TEST_CONTENT_ROOT=/root/ansible_collections/community/aws LC_ALL=en_US.UTF-8 /usr/bin/python3.8 /roo

markuman avatar Feb 10 '22 10:02 markuman

recheck

markuman avatar Mar 14 '22 19:03 markuman

Ah I think the terminator needs more permission to run this integration test

"An error occurred (AccessDeniedException) when calling the DeleteDocument operation: 

User: arn:aws:sts::966509639900:assumed-role/ansible-core-ci-test-prod/prod=remote=zuul-cloud is not authorized to perform:

ssm:DeleteDocument on resource: arn:aws:ssm:us-east-1:966509639900:document/ansible-custom-document because no identity-based policy allows the ssm:DeleteDocument action"

I made a PR for that https://github.com/mattclay/aws-terminator/pull/201

markuman avatar Mar 22 '22 10:03 markuman

hi @markuman, I already opened a MR for that in terminator: https://github.com/mattclay/aws-terminator/pull/192

blinkseb avatar Mar 22 '22 13:03 blinkseb

hi @markuman, I already opened a MR for that in terminator: mattclay/aws-terminator#192

oh sorry. I've closed mine.

markuman avatar Mar 22 '22 13:03 markuman

Helle everyone, is there something additional I need to do to have this PR merged? Thanks!

blinkseb avatar Oct 01 '22 07:10 blinkseb

Docs Build 📝

Thank you for contribution!✨

This PR has been merged and your docs changes will be incorporated when they are next published.

github-actions[bot] avatar Oct 18 '22 05:10 github-actions[bot]

Build failed.

:heavy_check_mark: ansible-galaxy-importer SUCCESS in 4m 44s :heavy_check_mark: build-ansible-collection SUCCESS in 5m 36s :x: ansible-test-sanity-docker-devel FAILURE in 10m 05s (non-voting) :heavy_check_mark: ansible-test-sanity-docker-milestone SUCCESS in 9m 37s :heavy_check_mark: ansible-test-sanity-docker-stable-2.12 SUCCESS in 10m 12s :heavy_check_mark: ansible-test-sanity-docker-stable-2.13 SUCCESS in 9m 32s :x: ansible-test-sanity-docker-stable-2.14 FAILURE in 9m 17s (non-voting) :heavy_check_mark: ansible-test-units-amazon-aws-python36 SUCCESS in 7m 04s :heavy_check_mark: ansible-test-units-amazon-aws-python38 SUCCESS in 5m 55s :heavy_check_mark: ansible-test-units-amazon-aws-python39 SUCCESS in 5m 43s :heavy_check_mark: ansible-test-splitter SUCCESS in 2m 43s :x: integration-community.aws-1 FAILURE in 6m 13s :warning: integration-community.aws-2 SKIPPED :warning: integration-community.aws-3 SKIPPED :warning: integration-community.aws-4 SKIPPED :warning: integration-community.aws-5 SKIPPED :warning: integration-community.aws-6 SKIPPED :warning: integration-community.aws-7 SKIPPED :warning: integration-community.aws-8 SKIPPED :warning: integration-community.aws-9 SKIPPED :warning: integration-community.aws-10 SKIPPED :warning: integration-community.aws-11 SKIPPED :warning: integration-community.aws-12 SKIPPED :warning: integration-community.aws-13 SKIPPED :warning: integration-community.aws-14 SKIPPED :warning: integration-community.aws-15 SKIPPED :warning: integration-community.aws-16 SKIPPED :warning: integration-community.aws-17 SKIPPED :warning: integration-community.aws-18 SKIPPED :heavy_check_mark: ansible-test-changelog SUCCESS in 2m 25s

recheck

alinabuzachis avatar Oct 18 '22 09:10 alinabuzachis

Build failed.

:heavy_check_mark: ansible-galaxy-importer SUCCESS in 3m 58s :heavy_check_mark: build-ansible-collection SUCCESS in 5m 17s :x: ansible-test-sanity-docker-devel FAILURE in 9m 55s (non-voting) :heavy_check_mark: ansible-test-sanity-docker-milestone SUCCESS in 10m 04s :heavy_check_mark: ansible-test-sanity-docker-stable-2.12 SUCCESS in 9m 58s :heavy_check_mark: ansible-test-sanity-docker-stable-2.13 SUCCESS in 8m 35s :x: ansible-test-sanity-docker-stable-2.14 FAILURE in 9m 17s (non-voting) :heavy_check_mark: ansible-test-units-amazon-aws-python36 SUCCESS in 6m 51s :heavy_check_mark: ansible-test-units-amazon-aws-python38 SUCCESS in 5m 13s :heavy_check_mark: ansible-test-units-amazon-aws-python39 SUCCESS in 6m 21s :heavy_check_mark: ansible-test-splitter SUCCESS in 2m 35s :x: integration-community.aws-1 FAILURE in 7m 30s :warning: integration-community.aws-2 SKIPPED :warning: integration-community.aws-3 SKIPPED :warning: integration-community.aws-4 SKIPPED :warning: integration-community.aws-5 SKIPPED :warning: integration-community.aws-6 SKIPPED :warning: integration-community.aws-7 SKIPPED :warning: integration-community.aws-8 SKIPPED :warning: integration-community.aws-9 SKIPPED :warning: integration-community.aws-10 SKIPPED :warning: integration-community.aws-11 SKIPPED :warning: integration-community.aws-12 SKIPPED :warning: integration-community.aws-13 SKIPPED :warning: integration-community.aws-14 SKIPPED :warning: integration-community.aws-15 SKIPPED :warning: integration-community.aws-16 SKIPPED :warning: integration-community.aws-17 SKIPPED :warning: integration-community.aws-18 SKIPPED :heavy_check_mark: ansible-test-changelog SUCCESS in 2m 46s

Okey, we still have CI permissions

User: arn:aws:sts::966509639900:assumed-role/ansible-core-ci-test-prod/prod=remote=zuul-cloud is not authorized to perform: ssm:DeleteDocument on resource: arn:aws:ssm:us-east-1:966509639900:document/ansible-custom-document because no identity-based policy allows the ssm:DeleteDocument action",

I think someone must review and merge https://github.com/mattclay/aws-terminator/pull/192 before we can get forward here.

markuman avatar Oct 18 '22 13:10 markuman

recheck

gravesm avatar Oct 19 '22 17:10 gravesm

Build failed.

:heavy_check_mark: ansible-galaxy-importer SUCCESS in 4m 00s :heavy_check_mark: build-ansible-collection SUCCESS in 4m 52s :x: ansible-test-sanity-docker-devel FAILURE in 10m 21s (non-voting) :heavy_check_mark: ansible-test-sanity-docker-milestone SUCCESS in 10m 17s :heavy_check_mark: ansible-test-sanity-docker-stable-2.12 SUCCESS in 10m 44s :heavy_check_mark: ansible-test-sanity-docker-stable-2.13 SUCCESS in 8m 40s :x: ansible-test-sanity-docker-stable-2.14 FAILURE in 9m 41s (non-voting) :heavy_check_mark: ansible-test-units-amazon-aws-python36 SUCCESS in 6m 26s :heavy_check_mark: ansible-test-units-amazon-aws-python38 SUCCESS in 5m 51s :heavy_check_mark: ansible-test-units-amazon-aws-python39 SUCCESS in 6m 59s :heavy_check_mark: ansible-test-splitter SUCCESS in 2m 26s :x: integration-community.aws-1 FAILURE in 44m 35s :warning: integration-community.aws-2 SKIPPED :warning: integration-community.aws-3 SKIPPED :warning: integration-community.aws-4 SKIPPED :warning: integration-community.aws-5 SKIPPED :warning: integration-community.aws-6 SKIPPED :warning: integration-community.aws-7 SKIPPED :warning: integration-community.aws-8 SKIPPED :warning: integration-community.aws-9 SKIPPED :warning: integration-community.aws-10 SKIPPED :warning: integration-community.aws-11 SKIPPED :warning: integration-community.aws-12 SKIPPED :warning: integration-community.aws-13 SKIPPED :warning: integration-community.aws-14 SKIPPED :warning: integration-community.aws-15 SKIPPED :warning: integration-community.aws-16 SKIPPED :warning: integration-community.aws-17 SKIPPED :warning: integration-community.aws-18 SKIPPED :heavy_check_mark: ansible-test-changelog SUCCESS in 2m 36s

recheck

markuman avatar Dec 07 '22 11:12 markuman

Build failed.

:heavy_check_mark: ansible-galaxy-importer SUCCESS in 4m 03s :heavy_check_mark: build-ansible-collection SUCCESS in 5m 36s :heavy_check_mark: ansible-test-sanity-docker-devel SUCCESS in 8m 55s (non-voting) :x: ansible-test-sanity-docker-milestone FAILURE in 8m 48s (non-voting) :heavy_check_mark: ansible-test-sanity-docker-stable-2.12 SUCCESS in 9m 41s :heavy_check_mark: ansible-test-sanity-docker-stable-2.13 SUCCESS in 9m 34s :heavy_check_mark: ansible-test-sanity-docker-stable-2.14 SUCCESS in 13m 16s :heavy_check_mark: ansible-test-units-amazon-aws-python36 SUCCESS in 7m 47s :heavy_check_mark: ansible-test-units-amazon-aws-python38 SUCCESS in 6m 12s :heavy_check_mark: ansible-test-units-amazon-aws-python39 SUCCESS in 6m 30s :heavy_check_mark: ansible-test-changelog SUCCESS in 2m 14s :heavy_check_mark: ansible-test-splitter SUCCESS in 3m 16s :x: integration-community.aws-1 FAILURE in 43m 48s :warning: integration-community.aws-2 SKIPPED :warning: integration-community.aws-3 SKIPPED :warning: integration-community.aws-4 SKIPPED :warning: integration-community.aws-5 SKIPPED :warning: integration-community.aws-6 SKIPPED :warning: integration-community.aws-7 SKIPPED :warning: integration-community.aws-8 SKIPPED :warning: integration-community.aws-9 SKIPPED :warning: integration-community.aws-10 SKIPPED :warning: integration-community.aws-11 SKIPPED :warning: integration-community.aws-12 SKIPPED :warning: integration-community.aws-13 SKIPPED :warning: integration-community.aws-14 SKIPPED :warning: integration-community.aws-15 SKIPPED :warning: integration-community.aws-16 SKIPPED :warning: integration-community.aws-17 SKIPPED :warning: integration-community.aws-18 SKIPPED :warning: integration-community.aws-19 SKIPPED :warning: integration-community.aws-20 SKIPPED :warning: integration-community.aws-21 SKIPPED :warning: integration-community.aws-22 SKIPPED

hmmm

"msg": "timed out waiting for ping module test: An error occurred (AccessDenied) when calling the PutObject operation: User: arn:aws:sts::966509639900:assumed-role/ansible-core-ci-test-prod/prod=remote=zuul-cloud is not authorized to perform: kms:GenerateDataKey on resource: arn:aws:kms:us-east-1:966509639900:key/5aa12287-6092-4fcc-9e8a-a08805e2706c because no identity-based policy allows the kms:GenerateDataKey action"

cc @gravesm

markuman avatar Dec 07 '22 12:12 markuman

Sure, would you mind submitting a PR for the missing policy to https://github.com/mattclay/aws-terminator?

gravesm avatar Dec 07 '22 15:12 gravesm

@blinkseb do you have some time to address the missing permission in a PR again?

markuman avatar Dec 07 '22 18:12 markuman

Build failed.

:heavy_check_mark: ansible-galaxy-importer SUCCESS in 5m 22s :heavy_check_mark: build-ansible-collection SUCCESS in 5m 34s :heavy_check_mark: ansible-test-sanity-docker-devel SUCCESS in 16m 30s (non-voting) :heavy_check_mark: ansible-test-sanity-docker-milestone SUCCESS in 13m 16s (non-voting) :heavy_check_mark: ansible-test-sanity-docker-stable-2.12 SUCCESS in 9m 53s :heavy_check_mark: ansible-test-sanity-docker-stable-2.13 SUCCESS in 9m 52s :heavy_check_mark: ansible-test-sanity-docker-stable-2.14 SUCCESS in 10m 36s :heavy_check_mark: ansible-test-units-amazon-aws-python36 SUCCESS in 8m 32s :heavy_check_mark: ansible-test-units-amazon-aws-python38 SUCCESS in 8m 28s :heavy_check_mark: ansible-test-units-amazon-aws-python39 SUCCESS in 7m 15s :heavy_check_mark: ansible-test-units-amazon-aws-python310 SUCCESS in 7m 35s :heavy_check_mark: ansible-test-changelog SUCCESS in 2m 17s :heavy_check_mark: ansible-test-splitter SUCCESS in 2m 32s :x: integration-community.aws-1 FAILURE in 13m 46s :x: integration-community.aws-2 FAILURE in 11m 11s :heavy_check_mark: integration-community.aws-3 SUCCESS in 5m 23s :warning: integration-community.aws-4 SKIPPED :warning: integration-community.aws-5 SKIPPED :warning: integration-community.aws-6 SKIPPED :warning: integration-community.aws-7 SKIPPED :warning: integration-community.aws-8 SKIPPED :warning: integration-community.aws-9 SKIPPED :warning: integration-community.aws-10 SKIPPED :warning: integration-community.aws-11 SKIPPED :warning: integration-community.aws-12 SKIPPED :warning: integration-community.aws-13 SKIPPED :warning: integration-community.aws-14 SKIPPED :warning: integration-community.aws-15 SKIPPED :warning: integration-community.aws-16 SKIPPED :warning: integration-community.aws-17 SKIPPED :warning: integration-community.aws-18 SKIPPED :warning: integration-community.aws-19 SKIPPED :warning: integration-community.aws-20 SKIPPED :warning: integration-community.aws-21 SKIPPED :warning: integration-community.aws-22 SKIPPED