aws_ssm connection plugin fails when used with Ansible 2.9.23

[MrBones757]

Summary

When running a script task (powershell script) against a windows host, an error is returned complaining about an invalid payload.

Issue Type

Bug Report

Component Name

aws_ssm.py script

Ansible Version

$ ansible --version
ansible 2.9.23.post0
  config file = None
  configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.8.6 (default, Jan 29 2021, 17:38:16) [GCC 8.4.1 20200928 (Red Hat 8.4.1-1)]

Collection Versions

$ ansible-galaxy collection list
Command did not work in AWX EE, from build though (latest for all):
collections:
  # Ansible
  - name: ansible.utils
  - name: awx.awx
  - name: community.general
  # AWS
  - name: community.aws
  - name: amazon.aws
  # VMware
  - name: community.vmware
  # Linux
  - name: ansible.posix
  # Windows
  - name: ansible.windows
  - name: community.windows
  - name: chocolatey.chocolatey

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /usr/local/lib/python3.8/site-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.18.9
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/local/lib/python3.8/site-packages
Requires: s3transfer, jmespath, botocore
Required-by: snowflake-connector-python
---
Name: botocore
Version: 1.21.9
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /usr/local/lib/python3.8/site-packages
Requires: python-dateutil, urllib3, jmespath
Required-by: s3transfer, boto3

Configuration

$ ansible-config dump --only-changed
<nothing>

OS / Environment

Custom AWX EE based on: 'docker.rwwa.com.au/ansible/ansible-runner:stable-2.9-devel'

Linux automation-job-64921-pshc9 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Steps to Reproduce

- name: Test The Thing
  hosts: "<windows-host-to-target"
  gather_facts: False
  tasks:
    - name: Run the Thing
      script: scripts/test.ps1

test.ps1 contains only "Write-Host 'hello word'" it is assumed the host vars of the targeted host, or some other source has the required parameters for connection plugin usage (credentials, bucket, region etc)

Expected Results

Script executes and job succeeds

Actual Results

{
  "changed": true,
  "rc": 1,
  "stdout": "invalid payload\nAt line:4 char:39\n+ If (-not $split_parts.Length -eq 2) { throw \"invalid payload\" }\n+                                       ~~~~~~~~~~~~~~~~~~~~~~~\n    + CategoryInfo          : OperationStopped: (invalid payload:String) [], RuntimeException\n    + FullyQualifiedErrorId : invalid payload\n \n",
  "stdout_lines": [
    "invalid payload",
    "At line:4 char:39",
    "+ If (-not $split_parts.Length -eq 2) { throw \"invalid payload\" }",
    "+                                       ~~~~~~~~~~~~~~~~~~~~~~~",
    "    + CategoryInfo          : OperationStopped: (invalid payload:String) [], RuntimeException",
    "    + FullyQualifiedErrorId : invalid payload",
    " "
  ],
  "stderr": "",
  "stderr_lines": [],
  "msg": "non-zero return code",
  "_ansible_no_log": false
}

Error is being thrown from: https://github.com/ansible/ansible/blob/2cbfd1e350cbe1ca195d33306b5a9628667ddda8/lib/ansible/executor/powershell/bootstrap_wrapper.ps1

this line

$split_parts = $exec_wrapper_str.Split(@("`0`0`0`0"), 2, [StringSplitOptions]::RemoveEmptyEntries)

appears to be the culprit, though im not really sure what this is trying to achieve.

If i enable connect debugging i see the below a few lines above the failure, along with some nomral encoded commands, not sure if this is the issue though. (could this be looking for a 4 element array but we are returning only 3 in aws_ssm or something like that)

(0, '', '')

Code of Conduct

• [X] I agree to follow the Ansible Code of Conduct

[jitterjuice]

Commenting as I wish to follow this issue :)

[ansibullbot]

cc @116davinder @jillr @markuman @mikedlr @nathanwebsterdotme @ozbillwang @s-hertel @tremble @wimnat click here for bot help

[116davinder]

@MrBones757, I believe, it's not related to aws_ssm.py module.

can you try to use https://docs.ansible.com/ansible/2.9/modules/win_shell_module.html windows specific module for this purpose?

[MrBones757]

Hi, I didn't see the response to this issue..

as per the documentation: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/script_module.html, the script module is supported on windows targets, so i don't believe that this is the root cause. I can confirm that win_shell worked in this case, i migrated to this as a workaround when working on this issue originally. I haven't been able to do any additional testing to see if other modules compatible with windows are affected by this when using this connection plugin.

As i understand it though, the error seems to be to do with how the response is being parsed from the connection plugin, that is, the PowerShell executor is receiving data from the aws_ssm connection plugin that is not valid.

we are in the process of moving to newer ansible versions (2.12), I intend on testing this issue again to see if the issue remains, or if it has been fixed somewhere along the line.

[tremble]

I'm sorry it's taken so long to respond to this issue.

Ansible 2.9 is no longer supported. Additionally a number of bugfixes have just landed with release 5.2.0 (including wrt Windows)

I'm going to close this issue. However if you're able to reproduce the issue with the newer versions of Ansible and the collection, please feel free to reopen it and we can try to figure out what's going on.