[stable29] fix: Make user removal more resilient

[backportbot[bot]]

Backport of #47896

Warning, This backport's changes differ from the original and might be incomplete ⚠️

Todo

• [ ] Review and resolve any conflicts
• [ ] Amend HEAD commit to remove the line stating to skip CI

---

Learn more about backports at https://docs.nextcloud.com/server/stable/go.php?to=developer-backports.

[susnux]

/cherry-pick 902fb8f9fc963701f3ecfb190a7dacc8eba43e54