maester No status indication of the M365 platform tests?

As the continuation from this thread: https://github.com/maester365/maester/issues/457

I wonder what I can do to ensure that these checks are executed successfully every day.

Some of the test has the indication when not executed. but not these tests:

As you can see the above, there is no status or even error thrown after the execution.

These tests (without the duplicate:

EIDSCA.AF02: Authentication Method - FIDO2 security key - Allow self-service set up.
EIDSCA.AF03: Authentication Method - FIDO2 security key - Enforce attestation.
EIDSCA.AF04: Authentication Method - FIDO2 security key - Enforce key restrictions.
EIDSCA.AF05: Authentication Method - FIDO2 security key - Restricted.
EIDSCA.AF06: Authentication Method - FIDO2 security key - Restrict specific keys.
EIDSCA.AT02: Authentication Method - Temporary Access Pass - One-time.
EIDSCA.CP01: Default Settings - Consent Policy Settings - Group owner consent for apps accessing data.
MS.AAD.4.1: Security logs SHALL be sent to the agency's security operations center for monitoring.
MS.EXO.1.1: Automatic forwarding to external domains SHALL be disabled.
MS.EXO.12.1: IP allow lists SHOULD NOT be created.
MS.EXO.12.2: Safe lists SHOULD NOT be enabled.
MS.EXO.13.1: Mailbox auditing SHALL be enabled.
MS.EXO.2.1: A list of approved IP addresses for sending mail SHALL be maintained.
MS.EXO.2.2: An SPF policy SHALL be published for each domain, designating only these addresses as approved senders.
MS.EXO.3.1: DKIM SHOULD be enabled for all domains.
MS.EXO.4.1: A DMARC policy SHALL be published for every second-level domain.
MS.EXO.4.2: The DMARC message rejection option SHALL be p=reject.
MS.EXO.4.3: The DMARC point of contact for aggregate reports SHALL include [email protected].
MS.EXO.5.1: SMTP AUTH SHALL be disabled.
MS.EXO.6.1: Contact folders SHALL NOT be shared with all domains.
MS.EXO.6.2: Calendar details SHALL NOT be shared with all domains.
MS.EXO.7.1: External sender warnings SHALL be implemented.
MS.EXO.8.1: A DLP solution SHALL be used.
MT.1002: App management restrictions on applications and service principals is configured and enabled.
MT.1021: Security Defaults are enabled.

Sep 16 '24 07:09 albert-widjaja

What do your Test details say? For the MS.EXO tests, it is probably due to not having Exchange Online connected.

Sep 16 '24 08:09 Haakonak

@Haakonak , Yes, it shows skipped, there is no status at the table, nor any icon. I assume it is expected this way.

As for the ExO testing, do I just follow this https://maester.dev/docs/installation/#installing-azure-and-exchange-online-modules and then manually click on the Run Maester Test button ?

Sep 16 '24 11:09 albert-widjaja

Hi @albert-widjaja

I think the exchange and/or azure part is not fully integrated, see here:

https://github.com/maester365/maester/blob/main/action.yml

Oct 07 '24 20:10 weyCC81

As for the ExO testing, do I just follow this https://maester.dev/docs/installation/#installing-azure-and-exchange-online-modules and then manually click on the Run Maester Test button ?

The critical piece would be to make sure you are using the -Service parameter that includes those when calling Connect-Maester or you can Invoke-Maester -SkipGraphConnect if you connect to these services prior to running the tests.

This demonstrates using all the services and connecting with appropriate credentials for the tests.

Oct 09 '24 01:10 soulemike

Hi @Snozzberries How can I get that going with the predefined "yml" and without cloning the repo and editing it?

https://maester.dev/docs/monitoring/github

Oct 10 '24 08:10 weyCC81

What do your Test details say? For the MS.EXO tests, it is probably due to not having Exchange Online connected. I believe the state attribute is wrong in my case, may this has changed recently?

Situation

Get-Module ExchangeOnlineManagement -ListAvailable # 3.6.0
Get-ConnectionInformation | ft Name, *state*, *token*
# Name             TokenExpiryTimeUTC         TokenStatus
# ----             ------------------         -----------
# ExchangeOnline_2 11.10.2024 12:01:26 +00:00 Active
# ExchangeOnline_3 11.10.2024 11:20:03 +00:00 Active
# ExchangeOnline_1 11.10.2024 09:35:48 +00:00 Active
(Get-ConnectionInformation | Where-Object { $_.Name -match 'ExchangeOnline' -and $_.state -eq 'Connected' })
# > Empty for me

Line: https://github.com/maester365/maester/blob/79e39fb5666b35c6e4595e50993b0410dfbb041b/powershell/public/core/Test-MtConnection.ps1#L49C13-L49C148

Workaround:

Remove the "ExchangeOnlineManagement" Module on the CurrentUser Scope (C:\Users\[CurrentUser]\Documents\WindowsPowerShell\Modules)
Update the "ExchangeOnlineManagement" Module from >3.0.0 to >3.4.0 (better 3.6.0) with Update-Module ExchangeOnlineManagement -Scope AllUsers
Try Maester again

Oct 10 '24 08:10 weyCC81

Hi @Snozzberries How can I get that going with the predefined "yml" and without cloning the repo and editing it?

https://maester.dev/docs/monitoring/github

The current action published on the Marketplace doesn't have compatibility with the other services. So you'd need to either handle them as custom steps in your own job, or you could look into adding more parameters to the Marketplace Action, but I am not sure if there'd be an elegant way due to the EXO module not using the Graph token.

https://github.com/maester365/maester/blob/main/action.yml

Ref #494

Oct 14 '24 01:10 soulemike

The exchange module supports an -AccessToken parameter. We should be able to get it from the existing connection and pass it through. @f-bader thoughts?

Oct 15 '24 09:10 merill

I have tried some variants, but have not yet managed to connect over federated credentials. Here are some ideas:

Az Module It does apparently not support an Exchange/Outlook scope

https://learn.microsoft.com/en-us/cli/azure/account?view=azure-cli-latest#az-account-get-access-token
https://learn.microsoft.com/de-de/powershell/module/az.accounts/get-azaccesstoken?view=azps-12.3.0#-resourcetypename

## $token = az account get-access-token --resource-type ms-graph
Connect-AzAccount -Identity
#$GraphToken = Get-AzAccessToken -ResourceTypeName MSGraph
$GraphToken = Get-AzAccessToken -ResourceTypeName [NoExchangeScopeFound]

Msal Module (Legacy)

# Install-Module -Name MSAL.PS -Scope AllUsers
Import-Module MSAL.PS
$connectionDetails = @{
    #'TenantId'     = 'contoso.onmicrosoft.com'
    'TenantId'     = '31537af4-6d77-4bb9-a681-d2394888ea26'
    'ClientId'     = 'f7bb0fcd-cedb-06d2-9ae6-0e287b347ff0'
    'ClientSecret' = '[Secret]' | ConvertTo-SecureString -AsPlainText -Force
}
$token2 = Get-MsalToken @connectionDetails -Scopes "https://outlook.office.com/.default"
$accessToken = $token2.AccessToken
$moera = "contoso.onmicrosoft.com"
Connect-ExchangeOnline -AccessToken $accessToken -Organization $moera -ShowBanner:$false

AzAuth Module

# Install-Module -Name AzAuth -Scope AllUsers
Import-Module AzAuth
$ConnectorArguments = @{
    ClientId = 'f7bb0fcd-cedb-06d2-9ae6-0e287b347ff0'
    Resource = 'https://outlook.office365.com/'
    TenantId = '31537af4-6d77-4bb9-a681-d2394888ea26'
    Scope = '.default' # this is the default, but I added it for clarification
    #Interactive = $true
}
$token3 = Get-AzToken @ConnectorArguments -ClientSecret $('[Secret]' | ConvertTo-SecureString -AsPlainText -Force)
$accessToken = $token3.Token
$moera = "contoso.onmicrosoft.com" 
Connect-ExchangeOnline -AccessToken $accessToken -Organization $moera -ShowBanner:$false

PS: Is it possible to make a pull request (PR) for outsider?

Oct 21 '24 22:10 weyCC81

I am not sure if there'd be an elegant way due to the EXO module not using the Graph token.

I should have been clearer in this statement. The Graph Module can support generating a token with the appropriate scopes, when you are using a service principal and the default scope. The necessary application scopes aren't assignable during an interactive request for the token. So it is possible to reuse that token from Graph, but you'd require a service principal rather than interactive. For the GitHub Action, that means it should work. With the main exception that the Security & Compliance module does not support an access token though.

Oct 21 '24 23:10 soulemike

Hi @albert-widjaja

I think the exchange and/or azure part is not fully integrated, see here:

https://github.com/maester365/maester/blob/main/action.yml

@weyCC81 , When will it be integrated with the main branch to give a wholistic view?

Oct 22 '24 08:10 albert-widjaja

Hi @albert-widjaja I think the exchange and/or azure part is not fully integrated, see here:

https://github.com/maester365/maester/blob/main/action.yml

@weyCC81 , When will it be integrated with the main branch to give a wholistic view?

As this is an Open Source Project, giving a timeline is probably not too easy. May you have an idea of how to solve the above challenges (I am just a commenter as you're in this project)?

Oct 22 '24 09:10 weyCC81

Hey @albert-widjaja, PR #505 is queued to add more clarity on how to use the source modules for authentication prior to running Invoke-Maester. That would be the short term answer for the GH Action, add the steps for authentication, then run Invoke-Maester, skipping Connect-Maester entirely. The SPN you are using for the OIDC federated authentication should still work fine for that purpose.

Oct 22 '24 14:10 soulemike