aap_callback failing to setup WinRM today

[UNiXMIT]

Summary

Yesterday (11th July 2023) my Ansible scripts were working OK. They use aap_callback to enable WinRM. The my script waits for port 5986 to become available using ansible.builtin.wait_for. Worked fine for months. Today (approx. lunchtime) it stopped working and my EC2 instances are no longer working with WinRM on port 5986. After my ansible script fails I try to telnet 5986 and sure enough it's not open. I can't understand what's changed.

Issue Type

Bug Report

Component Name

ec2_instance

Ansible Version

ansible [core 2.13.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/support/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /home/support/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.14 (main, Jan  9 2023, 00:00:00) [GCC 11.3.1 20220421 (Red Hat 11.3.1-2)]
  jinja version = 

Collection Versions

Collection                    Version
----------------------------- -------
amazon.aws                    5.4.0  
ansible.netcommon             3.1.0  
ansible.posix                 1.4.0  
ansible.utils                 2.6.1  
ansible.windows               1.11.0 
arista.eos                    5.0.1  
awx.awx                       21.4.0 
azure.azcollection            1.13.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.3.0  
cisco.aci                     2.2.0  
cisco.asa                     3.1.0  
cisco.dnac                    6.5.3  
cisco.intersight              1.0.19 
cisco.ios                     3.3.0  
cisco.iosxr                   3.3.0  
cisco.ise                     2.5.0  
cisco.meraki                  2.10.1 
cisco.mso                     2.0.0  
cisco.nso                     1.0.3  
cisco.nxos                    3.1.0  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.2  
community.aws                 3.5.0  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.5.0  
community.digitalocean        1.21.0 
community.dns                 2.3.1  
community.docker              2.7.1  
community.fortios             1.0.0  
community.general             5.5.0  
community.google              1.0.0  
community.grafana             1.5.2  
community.hashi_vault         3.2.0  
community.hrobot              1.5.2  
community.libvirt             1.2.0  
community.mongodb             1.4.2  
community.mysql               3.4.0  
community.network             4.0.1  
community.okd                 2.2.0  
community.postgresql          2.2.0  
community.proxysql            1.4.0  
community.rabbitmq            1.2.2  
community.routeros            2.2.1  
community.sap                 1.0.0  
community.sap_libs            1.2.0  
community.skydive             1.0.0  
community.sops                1.3.0  
community.vmware              2.8.0  
community.windows             1.11.0 
community.zabbix              1.8.0  
containers.podman             1.9.4  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.1  
dellemc.openmanage            5.5.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.19.0 
fortinet.fortimanager         2.1.5  
fortinet.fortios              2.1.7  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.8.1  
hpe.nimble                    1.1.4  
ibm.qradar                    2.0.0  
ibm.spectrum_virtualize       1.9.0  
infinidat.infinibox           1.3.3  
infoblox.nios_modules         1.3.0  
inspur.sm                     2.0.0  
junipernetworks.junos         3.1.0  
kubernetes.core               2.3.2  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.19.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.22.0
netapp.storagegrid            21.10.0
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.1  
netbox.netbox                 3.7.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.8.0  
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.2.3  
purestorage.flasharray        1.13.0 
purestorage.flashblade        1.9.0  
purestorage.fusion            1.0.2  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     2.0.0  
t_systems_mms.icinga_director 1.31.0 
theforeman.foreman            3.4.0  
vmware.vmware_rest            2.2.0  
vyos.vyos                     3.0.1  
wti.remote                    1.0.4 

AWS SDK versions

WARNING: Package(s) not found: boto
Name: boto3
Version: 1.26.118
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, s3transfer, botocore
Required-by: pfsso
---
Name: botocore
Version: 1.29.118
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: urllib3, jmespath, python-dateutil
Required-by: s3transfer, boto3

Configuration

No output shown

OS / Environment

cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="9.0 (Plow)" ID="rhel" ID_LIKE="fedora" VERSION_ID="9.0" PLATFORM_ID="platform:el9" PRETTY_NAME="Red Hat Enterprise Linux 9.0 (Plow)" ANSI_COLOR="0;31" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:redhat:enterprise_linux:9::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/9/" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 9" REDHAT_BUGZILLA_PRODUCT_VERSION=9.0 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="9.0"

Steps to Reproduce

- name: SSO 2FA
  hosts: local
  gather_facts: no
  tasks:
  - name: Import sso.yml
    import_tasks: sso.yml
  no_log: True
  tags: default

- name: AWS EC2 Management
  hosts: local
  vars:
    tempName: SEM-{{ 2048 | random }}
  gather_facts: no
  tasks:
  - name: Create AWS EC2 instance
    amazon.aws.ec2_instance:
      aap_callback:
        windows: true
        set_password: '{{ myPassword }}'
      name: '{{ tempName }}'
      image_id: '{{ awsAMI }}'
      key_name: '{{ keyName }}'
      network:  
        assign_public_ip: yes
      security_groups: '{{ securityGroups }}'
      region: '{{ awsRegion }}'
      availability_zone: '{{ availabilityZone }}'
      aws_profile: '{{ awsProfile }}'
      instance_type: '{{ instanceType }}'
      count: '{{ vmCount | default(1) }}'
      volumes:
      - device_name: '{{ deviceName }}'
        ebs:
         volume_type: '{{ volumeType }}'
         volume_size: '{{ volumeSize }}'
      state: running
      wait: true
    register: ec2
    tags: default
  
  - name: Wait 30 seconds for public IP and AAP Setup
    pause:
      seconds: 30
    tags: default

  - name: Add Host to awsEC2 Group 
    add_host: 
      hostname: '{{ item.public_ip_address }}'
      ansible_user: '{{ ansibleUser }}'
      ansible_password: '{{ myPassword }}'
      ansible_connection: winrm
      ansible_winrm_transport: basic
      ansible_winrm_server_cert_validation: ignore
      ansible_port: 5986
      ansible_winrm_operation_timeout_sec: 120
      ansible_winrm_read_timeout_sec: 140
      groups: awsEC2
    loop: '{{ ec2.instances }}'
    tags: default

  - name: Rename AWS EC2 instance
    amazon.aws.ec2_tag:
      aws_profile: '{{ awsProfile }}'
      region: '{{ awsRegion }}'
      resource: '{{ item }}'
      tags:
        Name: '{{ semaphore_vars.task_details.username|upper }}-{{ imageName }}'
        Owner: '{{ ssoEmail }}'
        OS: '{{ imageName }}'
      state: present
    loop: '{{ ec2.instance_ids }}'
    tags: default

  - name: WinRM Check
    ansible.builtin.wait_for:
      host: '{{ item.public_ip_address }}'
      port: 5986
      timeout: 120
    loop: '{{ ec2.instances }}'
    tags: default

Expected Results

Previously, before today, 'ansible.builtin.wait_for' would wait for port 5986 to become available and then the play would continue on successfully once it was available.

Actual Results

Timeout when waiting for x.x.x.x:5986

Code of Conduct

• [X] I agree to follow the Ansible Code of Conduct