Pester
Pester copied to clipboard
pester
Code coverage is not reported correctly in some circumstances
Checklist
- [X] Issue has a meaningful title
- [X] I have searched the existing issues. See all issues
- [X] I have tested using the latest version of Pester. See Installation and update guide.
What is the issue?
When running tests against a module script file that has a using module statement at the top of module script file in some cicumstances the code coverage is not reported accuratly what the tests actually covers.
There are 4 workarounds for this issue
- (non feasible) comment the line
using modulein SqlServerDsc.psm1 - (non feasible) change the order the tests are run
- switch to use the new code coverage method (not feasible yet due to https://github.com/pester/Pester/issues/2306).
- in the tests for
Assert-ManagedServiceTypeadd a mock forConvertFrom-ManagedServiceType.
But what I'm wondering is why I have to add the alternativ 4 workaround for it to work.
When running the code as written in the "How to reproduce" it does not cover the two line shown below. Doing any of the workarounds it will correctly cover 100%.
C:\source\DebugCodeCoverage2> Invoke-Pester -Configuration $pesterConfig
Pester v5.4.0
Starting discovery in 2 files.
Discovery found 4 tests in 33ms.
Starting code coverage.
Code Coverage preparation finished after 8 ms.
Running tests.
Running tests from 'C:\source\DebugCodeCoverage2\Assert-ManagedServiceType.Tests.ps1'
Describing Assert-ManagedServiceType
Context When types match
[+] Should not throw an exception 32ms (29ms|3ms)
Running tests from 'C:\source\DebugCodeCoverage2\ConvertFrom-ManagedServiceType.Tests.ps1'
Describing ConvertFrom-ManagedServiceType
Context When translating to a normalized service types
[+] Should properly map 'SqlServer' to normalized service type 'DatabaseEngine' 8ms (2ms|6ms)
[+] Should properly map 'SqlAgent' to normalized service type 'SqlServerAgent' 2ms (2ms|1ms)
[+] Should properly map 'Search' to normalized service type 'Search' 3ms (2ms|1ms)
Tests completed in 180ms
Tests Passed: 4, Failed: 0, Skipped: 0 NotRun: 0
Processing code coverage result.
Code Coverage result processed in 6 ms.
Covered 71.43% / 75%. 7 analyzed Commands in 1 File.
Missed commands:
File Class Function Line Command
---- ----- -------- ---- -------
SqlServerDsc.psm1 ConvertFrom-ManagedServiceType 55 $serviceTypeValue = 'SqlServerAgent'
SqlServerDsc.psm1 ConvertFrom-ManagedServiceType 62 $serviceTypeValue = 'Search'
Expected Behavior
The tests to pass and report 100% code coverage as the tests do individually.
Steps To Reproduce
File SqlServerDsc.psm1
# (non feasible) workaround 1 - Comment `using module` from SqlServerDsc.psm1
using module .\DependentClassModule.psm1
function Assert-ManagedServiceType
{
[OutputType()]
[CmdletBinding()]
param
(
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
[Object]
$ServiceObject,
[Parameter(Mandatory = $true)]
[ValidateSet('DatabaseEngine', 'SqlServerAgent', 'Search', 'IntegrationServices', 'AnalysisServices', 'ReportingServices', 'SQLServerBrowser', 'NotificationServices')]
[System.String]
$ServiceType
)
process
{
$normalizedServiceType = ConvertFrom-ManagedServiceType -ServiceType $ServiceObject.Type
if ($normalizedServiceType -ne $ServiceType)
{
# Removed to minimize code
}
}
}
function ConvertFrom-ManagedServiceType
{
[CmdletBinding()]
param
(
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
[System.String]
$ServiceType
)
process
{
switch ($ServiceType)
{
'SqlServer'
{
$serviceTypeValue = 'DatabaseEngine'
break
}
'SqlAgent'
{
$serviceTypeValue = 'SqlServerAgent'
break
}
'Search'
{
$serviceTypeValue = 'Search'
break
}
}
return $serviceTypeValue
}
}
File DependentClassModule.psm1
enum Ensure
{
Present
Absent
}
File Assert-ManagedServiceType.Tests.ps1
BeforeAll {
Import-Module -Name './SqlServerDsc.psm1' -Force
}
AfterAll {
# Unload the module being tested so that it doesn't impact any other tests.
Get-Module -Name 'SqlServerDsc' -All | Remove-Module -Force
}
Describe 'Assert-ManagedServiceType' -Tag 'Private' {
BeforeAll {
$mockServiceObject = @{
Type = 'SqlServer'
}
}
Context 'When types match' {
# ------
# Workaround 3 - In tests for `Assert-ManagedServiceType` add a mock for `ConvertFrom-ManagedServiceType`.
# ------
# BeforeAll {
# Mock -CommandName ConvertFrom-ManagedServiceType -ModuleName 'SqlServerDsc' -MockWith {
# return 'DatabaseEngine'
# }
# }
It 'Should not throw an exception' {
$_.MockServiceObject = $mockServiceObject
InModuleScope -Parameter $_ -ModuleName 'SqlServerDsc' -ScriptBlock {
Set-StrictMode -Version 1.0
{
$MockServiceObject |
Assert-ManagedServiceType -ServiceType 'DatabaseEngine' -ErrorAction 'Stop'
} | Should -Not -Throw
}
}
}
}
File ConvertFrom-ManagedServiceType.Tests.ps1
BeforeAll {
Import-Module -Name './SqlServerDsc.psm1' -Force
}
AfterAll {
# Unload the module being tested so that it doesn't impact any other tests.
Get-Module -Name 'SqlServerDsc' -All | Remove-Module -Force
}
Describe 'ConvertFrom-ManagedServiceType' -Tag 'Private' {
Context 'When translating to a normalized service types' {
BeforeDiscovery {
$testCases = @(
@{
MockServiceType = 'SqlServer'
MockExpectedType = 'DatabaseEngine'
}
@{
MockServiceType = 'SqlAgent'
MockExpectedType = 'SqlServerAgent'
}
@{
MockServiceType = 'Search'
MockExpectedType = 'Search'
}
)
}
It 'Should properly map ''<MockServiceType>'' to normalized service type ''<MockExpectedType>''' -ForEach $testCases {
InModuleScope -Parameters $_ -ModuleName 'SqlServerDsc' -ScriptBlock {
Set-StrictMode -Version 1.0
# Get the ManagedServiceType
$managedServiceType = ConvertFrom-ManagedServiceType -ServiceType $MockServiceType
$managedServiceType | Should -BeOfType [System.String]
$managedServiceType | Should -Be $MockExpectedType
}
}
}
}
File debug.ps1
$pesterConfig = New-PesterConfiguration -Hashtable @{
CodeCoverage = @{
Enabled = $true
Path = './SqlServerDsc.psm1'
OutputPath = './Pester_coverage.xml'
UseBreakpoints = $true
}
Run = @{
Path = @(
# (non feasible) workaround 2 - Change the order the tests are run
# Change order and the coverage become 100%
'.\Assert-ManagedServiceType.Tests.ps1'
'.\ConvertFrom-ManagedServiceType.Tests.ps1'
)
}
Output = @{
Verbosity = 'Detailed'
}
}
Invoke-Pester -Configuration $pesterConfig
Describe your environment
Pester version : 5.4.0 C:\source\SqlServerDsc\output\RequiredModules\Pester\5.4.0\Pester.psm1
PowerShell version : 7.3.2
OS version : Microsoft Windows NT 10.0.22623.0
Possible Solution?
Unknown :/
Thanks for the report! I'm able to reproduce this and it's a problem with PowerShell.
I have no idea why using module would cause the breakpoint actions not to run, but we'll have to add it to the long list of weird class behavior. 😞 Would you mind posting an issue in the PowerShell-repo or should I?
Replace debug.ps1 with the following for a neutral repro:
Get-PSBreakpoint | Remove-PSBreakpoint
$null = Set-PSBreakpoint -Script ./SqlServerDsc.psm1 -Line 20 -Action { Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)" }
$null = Set-PSBreakpoint -Script ./SqlServerDsc.psm1 -Line 38 -Action { Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)" }
$null = Set-PSBreakpoint -Script ./SqlServerDsc.psm1 -Line 40 -Action { Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)" }
$null = Set-PSBreakpoint -Script ./SqlServerDsc.psm1 -Line 46 -Action { Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)" }
$null = Set-PSBreakpoint -Script ./SqlServerDsc.psm1 -Line 52 -Action { Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)" }
$null = Set-PSBreakpoint -Script ./SqlServerDsc.psm1 -Line 58 -Action { Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)" }
$null = Set-PSBreakpoint -Script ./SqlServerDsc.psm1 -Line 22 -Action { Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)" }
Get-PSBreakpoint
Import-Module -Name './SqlServerDsc.psm1' -Force
& (Get-Module SqlServerDsc) { ConvertFrom-ManagedServiceType -ServiceType 'SqlServer' }
Get-Module -Name 'SqlServerDsc' -All | Remove-Module -Force
Get-PSBreakpoint
Import-Module -Name './SqlServerDsc.psm1' -Force
& (Get-Module SqlServerDsc) { ConvertFrom-ManagedServiceType -ServiceType 'SqlServer' }
& (Get-Module SqlServerDsc) { ConvertFrom-ManagedServiceType -ServiceType 'SqlAgent' }
& (Get-Module SqlServerDsc) { ConvertFrom-ManagedServiceType -ServiceType 'Search' }
Get-Module -Name 'SqlServerDsc' -All | Remove-Module -Force
Output:
ID Script Line Command Variable Action
-- ------ ---- ------- -------- ------
344 SqlServerDsc.psm1 20 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
345 SqlServerDsc.psm1 38 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
346 SqlServerDsc.psm1 40 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
347 SqlServerDsc.psm1 46 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
348 SqlServerDsc.psm1 52 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
349 SqlServerDsc.psm1 58 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
350 SqlServerDsc.psm1 22 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
Hello world from 345 on line 38:0
Hello world from 346 on line 40:0
Hello world from 349 on line 58:0
DatabaseEngine
# All breakpoints still exist, but none are triggered after the second module import
ID Script Line Command Variable Action
-- ------ ---- ------- -------- ------
344 SqlServerDsc.psm1 20 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
345 SqlServerDsc.psm1 38 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
346 SqlServerDsc.psm1 40 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
347 SqlServerDsc.psm1 46 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
348 SqlServerDsc.psm1 52 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
349 SqlServerDsc.psm1 58 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
350 SqlServerDsc.psm1 22 Write-Host "Hello world from $($_.Id) on line $($_.Line):$($_.Column)"
DatabaseEngine
SqlServerAgent
Search
Would you mind posting an issue in the PowerShell-repo or should I?
@fflaten if you have time I would much appreciate if you could do it. 🙇 But since there is a workaround for issue #2306 this issue no longer as important as the solution for this issue is to move to the new code coverage method (UseBreakpoints = $false). 🤔