PerfView & PerfViewCollect not working in a Windows Server Core k8s pod

[ysiivan]

Sysinfo

WindowsBuildLabEx                                       : 18362.1.amd64fre.19h1_release.190318-1202
WindowsCurrentVersion                                   : 6.3
WindowsEditionId                                        : ServerDatacenterACor
WindowsInstallationType                                 : Server Core
WindowsInstallDateFromRegistry                          : 8/6/2020 4:28:25 PM
WindowsProductId                                        : 00431-40000-00000-AA301
WindowsProductName                                      : Windows Server Datacenter
WindowsRegisteredOrganization                           :
WindowsRegisteredOwner                                  :
WindowsSystemRoot                                       : C:\Windows
WindowsVersion                                          : 1903
BiosCharacteristics                                     : {4, 7, 9, 11...}
BiosBIOSVersion                                         : {INTEL  - 6040000, VMW71.00V.17369862.B64.2012240522,
                                                          VMware, Inc. - 10000}

Problem (PerfView)

[EXECUTING: PerfView /logFile=log.txt /maxCollectSec=60 collect]
Circular buffer size = 0, setting to 500.
[Kernel Log: C:\diag\output\PerfViewData.kernel.etl]
Kernel keywords enabled: Default
Aborting tracing for sessions 'NT Kernel Logger' and 'PerfViewSession'.
Ensuring .NET Allocation profiler not installed.
Detected 64 bit system, removing 32 bit keys.
[Exception Occurred: System.Runtime.InteropServices.COMException (0x80070522): A required privilege is not held by the c
lient. (Exception from HRESULT: 0x80070522)
   at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
   at Microsoft.Diagnostics.Tracing.Session.TraceEventSession.EnableKernelProvider(Keywords flags, Keywords stackCapture
)
   at PerfView.CommandProcessor.Start(CommandLineArgs parsedArgs)
   at PerfView.CommandProcessor.Collect(CommandLineArgs parsedArgs)
   at PerfView.CommandProcessor.ExecuteCommand(CommandLineArgs parsedArgs)]
[DONE 16:16:00 FAIL: PerfView /logFile=log.txt /maxCollectSec=60 collect]

Problem (PerfViewCollect)

PS C:\diag\output> ..\pvc\PerfViewCollect.exe /MaxCollectSec:30 /LogFile:log.txt collect

Pre V4.0 .NET Rundown disabled, Type 'E' to enable symbols for V3.5 processes.
Type S to stop collection, 'A' will abort.
PS C:\diag\output> cat .\log.txt
PerfView logging started at 1/18/2023 5:21:22 PM
[EXECUTING: PerfView /MaxCollectSec:30 /LogFile:log.txt collect]
Circular buffer size = 0, setting to 500.
[Kernel Log: C:\diag\output\PerfViewData.kernel.etl]
Kernel keywords enabled: Default
Aborting tracing for sessions 'NT Kernel Logger' and 'PerfViewSession'.
Ensuring .NET Allocation profiler not installed.
Detected 64 bit system, removing 32 bit keys.
[Exception Occurred: System.Runtime.InteropServices.COMException (0x80070522): A required privilege is not held by the c
lient. (0x80070522)
   at System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(Int32 errorCode)
   at Microsoft.Diagnostics.Tracing.Session.TraceEventSession.EnableKernelProvider(Keywords flags, Keywords stackCapture
) in C:\work\apg\perfview\src\TraceEvent\TraceEventSession.cs:line 734
   at PerfView.CommandProcessor.Start(CommandLineArgs parsedArgs) in C:\work\apg\perfview\src\PerfView\CommandProcessor.
cs:line 502
   at PerfView.CommandProcessor.Collect(CommandLineArgs parsedArgs) in C:\work\apg\perfview\src\PerfView\CommandProcesso
r.cs:line 208
   at PerfView.CommandProcessor.ExecuteCommand(CommandLineArgs parsedArgs) in C:\work\apg\perfview\src\PerfView\CommandP
rocessor.cs:line 38]
[DONE 17:21:22 FAIL: PerfView /MaxCollectSec:30 /LogFile:log.txt collect]

User

PS C:\diag\output> whoami
user manager\containeradministrator
PS C:\diag\output>
PS C:\diag\output> $currentPrincipal = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentit
y]::GetCurrent())
PS C:\diag\output> $currentPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
True

[brianrob]

Are you using process isolation containers? If so, this error is expected, because the kernel is shared amongst all of the containers on the machine, and allowing those events to flow into the container would leak information from outside the container.

The recommendation here is to capture a trace on the host with /EnableEventsInContainers, and then copy the trace into the container and run PerfView merge -ImageIDsOnly against the trace. This will ensure that you get kernel events, and also that your stacks are resolvable.