🪲 CISA.MS.AAD.3.3 evaluates to "failed" although all recommended settings are set

[GeldHades27355]

Describe the bug

CISA.MS.AAD.3.3 evaluates to "failed" on our tenant, although all settings are set as recommended:

1. Show app name is ON
2. Show geographic location is ON
3. Authentication methods migration is COMPLETE/PASS

To Reproduce

1. Install Maester 1.3.0
2. Pester 5.5.0 autoinstalls
3. connect-maester (phishing resistant!)
4. connect-azaccount -tenant -authscope
5. invoke-maester
6. observe CISA.MS.AAD.3.3 is FAILED

Expected behavior

CISA.MS.AAD.3.3 should be PASSED

Environment Data

Name Value

---

PSVersion 7.5.2 PSEdition Core GitCommitId 7.5.2 OS Microsoft Windows 10.0.26100 Platform Win32NT PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…} PSRemotingProtocolVersion 2.3 SerializationVersion 1.1.0.1 WSManStackVersion 3.0 System Arm64 Tenant license M365 Business Premium + Entra P2

[soulemike]

Hey @GeldHades27355, this was functioning correctly from what I could tell, but this single test does require a half dozen or so settings to be properly configured for it to pass. I submitted a PR to improve the test results to better demonstrate if there is a setting causing the test not to pass.

[GeldHades27355]

Half a dozen? Maester suggests precisely 3 settings, which are all set accordingly. Which others settings does this need to pass?

[soulemike]

1. Authentication Methods Migration Complete
2. Microsoft Authenticator state is enabled
3. Microsoft Authenticator Included Targets is all users
4. Allow use of Microsoft Authenticator OTP is disabled
5. Show application name in push and passwordless notifications status is enabled
6. Show application name in push and passwordless notifications included target is all users
7. Show geographic location in push and passwordless notifications status is enabled
8. Show geographic location in push and passwordless notifications included target is all users

[GeldHades27355]

Ah I see. We only target guest users with Authenticator, as we enforce phishing-resistant MFA for all internal folks on all tenants we manage.

Considering that we're stricter than the recommendation, what is the rationale for penalizing tenants that basically made Authenticator obsolete?

[soulemike]

The Maester tests just map to the requirements from the upstream benchmark controls when there is a mapping, in this case CISA's recommendations.

Each identifier clicks through to their rationale. https://maester.dev/docs/tests/cisa/entra

Though, this type of question does come up often enough I have it on my to-do list to write up a blog post on compensating controls.

[GeldHades27355]

Of course. A look at https://github.com/cisagov/ScubaGear/blob/main/PowerShell/ScubaGear/baselines/aad.md#msaad33v2

Doesn't seem to mention any of the other dependencies you previously listed.

So yes, Authenticator is enabled for guest users. Yes, it is configured to show login context info (see OP).

Here's why we turn it off for users: We don't want to suggest it is valid method to authenticate when adding new authentication methods via self service. As you know, Entra will let you configure a method that does not actually work for authentication. To reduce service tickets due to questions and confusion, we only enable authentication methods that satisfy phishing resistant MFA: FIDO2 and Windows Hello.

I understand that you'd like to be as aligned with standard recommendations for compliance reasons, but there should be a way to "pass" items that exceed standard recommendations.

Or, implement a forth outcome:

1. Fail
2. Pass / CISA compliant (in this case)
3. Skipped
4. Alternate implementation (Not CISA compliant, but equivalent or better)

[soulemike]

Here is CISA's implementation, which you can see the same requirements. https://github.com/cisagov/ScubaGear/blob/30a78bcd52390883a3f5167fcbfe9a546b161ca4/PowerShell/ScubaGear/Rego/AADConfig.rego#L282

The current method with Maester to handle what you are looking for would be to remove, skip, or ignore the default test and create your own custom test that validated the control state for your specifications to meet your policy.

[GeldHades27355]

Ah - thanks. Yeah, I can see that now.

We'll think about custom tests, if we can mobilize the resources for them. Thanks again!

[soulemike]

Just as an aside to the idea above, we published a blog around controls. https://maester.dev/blog/compensating-controls