ansible.windows icon indicating copy to clipboard operation
ansible.windows copied to clipboard

Published 20 hours ago • verified publisher icon ansible-collections

Windows update failes due to update loop

Open Milosz-Galecki-wttech opened this issue 2 months ago • 1 comments

SUMMARY

I'm trying to update two freshly installed Windows 2022 servers, but it keeps failing on the same update KB5034439

ISSUE TYPE
  • Bug Report
COMPONENT NAME

win_updates

ANSIBLE VERSION
ansible [core 2.15.10]
  config file = None
  configured module search path = ['/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.18 (main, Jan 24 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.3
  libyaml = True
COLLECTION VERSION
Collection      Version
--------------- -------
ansible.windows 2.3.0
OS / ENVIRONMENT

Windows Server 2022 Standard

STEPS TO REPRODUCE

Install new Windows Server 2022 Standard and try to patch it to the latest state via ansible.

        - name: Apply updates
          ansible.windows.win_updates:
            category_names: '*'
            reboot: true
            log_path: c:\temp\patching.txt
            state: installed
EXPECTED RESULTS

All available patches get installed successfully

ACTUAL RESULTS

Update failes

TASK [Apply updates] ***********************************************************
task path: /runner/project/win-update.yml:39
<10.21.38.15> Running win_updates - round 1
<10.21.38.15> Starting update task
<10.21.38.11> Running win_updates - round 1
<10.21.38.11> Starting update task
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
EXEC (via pipeline wrapper)
EXEC (via pipeline wrapper)
<10.21.38.11> Starting polling for update results
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.15> Starting polling for update results
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.11> Download progress - Total: 23086576/23086576 100%, Update (2024-01 Security Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5034439)): 23086576/23086576 100%, Phase: Downloading
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.11> Update phase download completed
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.15> Download progress - Total: 23086576/23086576 100%, Update (2024-01 Security Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5034439)): 23086576/23086576 100%, Phase: Downloading
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.15> Update phase download completed
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.11> Install progress - Total: 100%, Update (2024-01 Security Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5034439)): 100%
<10.21.38.11> Update phase install completed
<10.21.38.11> Received final progress result from update task
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.11> Failure when running win_updates module (Will retry after reboot): Failed to install all updates - see updates for more information
<10.21.38.11> Rebooting host after installing updates
EXEC (via pipeline wrapper)
ansible.windows.win_updates: rebooting server...
EXEC (via pipeline wrapper)
ansible.windows.win_updates validating reboot
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
EXEC (via pipeline wrapper)
EXEC (via pipeline wrapper)
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> Install progress - Total: 100%, Update (2024-01 Security Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5034439)): 100%
<10.21.38.15> Update phase install completed
<10.21.38.15> Received final progress result from update task
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.15> Failure when running win_updates module (Will retry after reboot): Failed to install all updates - see updates for more information
<10.21.38.15> Rebooting host after installing updates
EXEC (via pipeline wrapper)
ansible.windows.win_updates: rebooting server...
EXEC (via pipeline wrapper)
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
ansible.windows.win_updates validating reboot
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
EXEC (via pipeline wrapper)
EXEC (via pipeline wrapper)
EXEC (via pipeline wrapper)
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.11
EXEC (via pipeline wrapper)
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
ansible.windows.win_updates running post reboot test command
EXEC (via pipeline wrapper)
ansible.windows.win_updates: system successfully rebooted
<10.21.38.11> Running win_updates - round 2
<10.21.38.11> Starting update task
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.11> Starting polling for update results
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.15> ESTABLISH WINRM CONNECTION FOR USER: xxxxxxxxx on PORT 5985 TO 10.21.38.15
EXEC (via pipeline wrapper)
ansible.windows.win_updates running post reboot test command
EXEC (via pipeline wrapper)
ansible.windows.win_updates: system successfully rebooted
<10.21.38.15> Running win_updates - round 2
<10.21.38.15> Starting update task
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.15> Starting polling for update results
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
<10.21.38.11> Install progress - Total: 100%, Update (2024-01 Security Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5034439)): 100%
<10.21.38.11> Update phase install completed
<10.21.38.11> Received final progress result from update task
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
fatal: [10.21.38.11]: FAILED! => {
    "changed": true,
    "failed_update_count": 1,
    "filtered_updates": {},
    "found_update_count": 1,
    "installed_update_count": 0,
    "invocation": {
        "module_args": {
            "accept_list": null,
            "category_names": [
                "*"
            ],
            "log_path": "c:\\\\temp\\\\patching.txt",
            "reboot": true,
            "reboot_timeout": 1200,
            "reject_list": null,
            "server_selection": "default",
            "skip_optional": false,
            "state": "installed"
        }
    },
    "msg": "An update loop was detected, this could be caused by an update being rolled back during a reboot or the Windows Update API incorrectly reporting a failed update as being successful.Check the Windows Updates logs on the host to gather more information. Updates in the reboot loop are: a8a2d6e3-c6dc-4eb8-bcfb-8c8c7d947899",
    "reboot_required": false,
    "rebooted": true,
    "updates": {
        "a8a2d6e3-c6dc-4eb8-bcfb-8c8c7d947899": {
            "categories": [
                "Microsoft Server operating system-21H2",
                "Security Updates"
            ],
            "downloaded": true,
            "failure_hresult_code": -1,
            "failure_msg": "Unknown WUA HRESULT -1 (UNKNOWN 0xFFFFFFFF)",
            "id": "a8a2d6e3-c6dc-4eb8-bcfb-8c8c7d947899",
            "installed": false,
            "kb": [
                "5034439"
            ],
            "title": "2024-01 Security Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5034439)"
        }
    }
}
<10.21.38.15> Install progress - Total: 100%, Update (2024-01 Security Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5034439)): 100%
<10.21.38.15> Update phase install completed
<10.21.38.15> Received final progress result from update task
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_updates.ps1
Pipelining is enabled.
EXEC (via pipeline wrapper)
fatal: [10.21.38.15]: FAILED! => {
    "changed": true,
    "failed_update_count": 1,
    "filtered_updates": {},
    "found_update_count": 1,
    "installed_update_count": 0,
    "invocation": {
        "module_args": {
            "accept_list": null,
            "category_names": [
                "*"
            ],
            "log_path": "c:\\\\temp\\\\patching.txt",
            "reboot": true,
            "reboot_timeout": 1200,
            "reject_list": null,
            "server_selection": "default",
            "skip_optional": false,
            "state": "installed"
        }
    },
    "msg": "An update loop was detected, this could be caused by an update being rolled back during a reboot or the Windows Update API incorrectly reporting a failed update as being successful.Check the Windows Updates logs on the host to gather more information. Updates in the reboot loop are: a8a2d6e3-c6dc-4eb8-bcfb-8c8c7d947899",
    "reboot_required": false,
    "rebooted": true,
    "updates": {
        "a8a2d6e3-c6dc-4eb8-bcfb-8c8c7d947899": {
            "categories": [
                "Microsoft Server operating system-21H2",
                "Security Updates"
            ],
            "downloaded": true,
            "failure_hresult_code": -1,
            "failure_msg": "Unknown WUA HRESULT -1 (UNKNOWN 0xFFFFFFFF)",
            "id": "a8a2d6e3-c6dc-4eb8-bcfb-8c8c7d947899",
            "installed": false,
            "kb": [
                "5034439"
            ],
            "title": "2024-01 Security Update for Microsoft server operating system version 21H2 for x64-based Systems (KB5034439)"
        }
    }
}

Milosz-Galecki-wttech avatar Apr 18 '24 08:04 Milosz-Galecki-wttech

Unfortunately there is not much we can do at this point. What is happening is:

  • The win_updates API find update KB5034439 as an update ready to install
  • It is installed
  • The host is rebooted
  • Next round it finds the same update as ready to install
  • It is installed
  • The host is rebooted
  • Infinitely repeats

Something is causing the update to rollback on the reboot stage so when we go to check subsequent updates to install it is found as needed and it is installed again. To avoid hanging the task forever we have a check after each install attempt post reboot to see if it installed the same updates as before. The module has no oversight over the rollback stage, it can only see the update is ready to be installed.

The only recourse for yourself here is

  • Look into the Windows Updates logs Get-WindowsUpdateLog (and DISM logs) to see why it is being rolled back
  • Exclude the update temporarily
  • Not use reboot: True so this loop doesn't happen
    • This doesn't fix the problem as the next reboot will roll it back anyway

None of the options are ideal but our hands are tied by the API that Microsoft exposes here.

jborean93 avatar Apr 19 '24 00:04 jborean93