fleet icon indicating copy to clipboard operation
fleet copied to clipboard

Windows 11 Hotpatching Provider Overrides Fleet OS Update Policies

Open AdamBaali opened this issue 1 month ago • 7 comments

💥 Actual behavior

On specific Windows 11 Enterprise configurations, Fleet's OS Update policies (deadlines and grace periods) are ignored.

The Windows Hotpatching provider (GUID B04F44A4-B696-4B56-934A-C11667E944E4) claims "Winning Provider" status in the registry. Because this provider doesn't handle full MDM management, QualityUpdateEnrolled defaults to 0, causing the Windows Update Agent to ignore Fleet's configuration.

🛠️ To fix

Up to product team.... Suggestion to look at:

Fleet could detect this conflict and assert priority, or automatically disable the Hotpatching feature if it interferes with managed updates.

Workaround: Explicitly disable Hotpatching via the Policy Manager (x64 compatible): HKLM\SOFTWARE\Microsoft\PolicyManager\current\device\Update Value: AllowRebootlessUpdates = 0 (DWORD)

(Note: The previously suggested HotPatchRestrictions key is specific to Arm64 hardware and is ignored by the OS on x64 clients).

🧑‍💻 Steps to reproduce

These steps:

  • [ ] Have been confirmed to consistently lead to reproduction in multiple Fleet instances.
  • [x] Describe the workflow that led to the error, but have not yet been reproduced in multiple Fleet instances.
  1. Enroll a Windows 11 Enterprise device with Hotpatching features (AllowRebootlessUpdates) active.
  2. Apply a Windows Update profile via Fleet (e.g. set a deadline).
  3. Check the registry at HKLM\SOFTWARE\Microsoft\PolicyManager\current\device\Update.
  4. Observe that QualityUpdateEnrolled is 0 and WinningProvider matches the Hotpatch GUID (B04F...).

🕯️ More info (optional)

This appears to be an arbitration conflict where the OS favors the native "Rebootless Updates" provider over the MDM provider. The registry keys for the Hotpatch provider regenerate immediately if deleted manually.

AdamBaali avatar Dec 02 '25 16:12 AdamBaali

@AdamBaali Is the Hotpatching feature enabled on every Windows host by default? Is this something that was left from the previous MDM solution?

Can a script solve disabling hotpatching so it makes Fleet's update configuration active again?

marko-lisica avatar Dec 03 '25 12:12 marko-lisica

@marko-lisica I think it was something from the previous MDM solution I will speak with customer-deebradel and see if the suggested script is working today as we've been trying many different things without much success to remove GUID B04F44A4-B696-4B56-934A-C11667E944E4

AdamBaali avatar Dec 03 '25 13:12 AdamBaali

Update on investigation and scope:

Critical Scope Update: We have confirmed that this issue is not limited to devices migrated from Intune or those previously running N-able/N-sight agents. We are now observing the exact same behavior on freshly enrolled devices (non-Intune, non-N-able) enrolled directly into Fleet.

Symptoms:

  • QualityUpdateEnrolled remains at 0 in HKLM\SOFTWARE\Microsoft\PolicyManager\current\device\Update.
  • The "Winning Provider" for update policies is consistently overridden by the system GUID B04F44A4-B696-4B56-934A-C11667E944E4 (identified as the Windows UpdatePolicy / Hotpatch enrollment), rather than the Fleet MDM enrollment.
  • The Fleet enrollment GUID appears to be in an error state (EnrollmentState: 3) rather than active (EnrollmentState: 1) on affected devices, despite CSP keys being present.

Summary of Remediation Attempts (Unsuccessful):

  1. Manual Registry Intervention: Manually setting QualityUpdateEnrolled to 1 via PowerShell. The value is immediately reverted by the system (Winning Provider B04F...).
  2. Third-Party Agent Removal: Full removal of N-able/N-sight agents and services to rule out interference.
  3. GPO Cleanup: Removed legacy Windows Update GPO keys and cache (HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate).
  4. Provider Cleanup Scripts: Attempted to script the removal of the conflicting B04F... GUID from Enrollments, PolicyManager\Providers, and Provisioning. The registry keys regenerate immediately or upon reboot.
  5. Hotpatching Disablement: Attempted to disable Windows 11 Hotpatching by setting AllowRebootlessUpdates to 0 and removing the B04F... provider. The registry state rebuilds itself after a reboot, reverting QualityUpdateEnrolled to 0.

customer-deebradel has an impacted device available with remote access to facilitate further debugging.

AdamBaali avatar Dec 04 '25 16:12 AdamBaali

@lukeheath We believe this might be a P2 bug as it is currently preventing customer-deebradel from using Fleet's software update enforcement feature.

It was originally thought this was caused by a previous 3rd party MDM enrollment or patching provider, but they are seeing it on brand new computers that have only ever been enrolled into Fleet.

ddribeiro avatar Dec 04 '25 17:12 ddribeiro

  • @georgekarrv for the p2 request @ddribeiro fyi that EM's are now the dri's for p labeling bug reports

zayhanlon avatar Dec 04 '25 17:12 zayhanlon

So this only fails with 'hotpatcching enabled'? Did we ever claim to support hotpatching? This does sound like a feature request more than a bug.

I agree we should understand what is causing the issue so an investigation is warranted.

@JordanMontgomery can you try to triage this?

georgekarrv avatar Dec 10 '25 16:12 georgekarrv

I've spent a bit of time over the past couple days trying to triage this. I have been able to get the "Cloud" autopatch provider activated by joining a sysetm to Entra and Intune before migrating it too fleet. Even so, when I do this, after letting the device sit overnight and with the QualityUpdateEnrolled setting above set to 0, my host is still enforcing the Fleet update CSP. I note that autopatch settings get referred to in the UI as "Managed Quality updates", "Managed Feature updates", etc which I think lines up with the behavior of Autopatch but when disabled it means that other means(e.g. update CSP) are used to manage updating. Autopatch does take a more direct appraoch of targeting specific systems with specific updates from the Cloud rather than the update CSP's rule based approach which basically just tells the system to install updates of a specified type within a specified number of days.

Image Image

I think this issue may have been further complicated by the recently discovered https://github.com/fleetdm/fleet/issues/37137 which caused update CSP not to be deployed in some cases when customers manage their fleet instances via Gitops

JordanMontgomery avatar Dec 12 '25 15:12 JordanMontgomery