microsoft
Running ARI on Azure Runbook got error "The command cannot remove the job with the job ID 10 and the name ResourceJob_Compute because the job is not finished. To remove the job, first stop the job, or use the Force parameter"
Runbook
Import-Module AzureResourceInventory
Import-Module ThreadJob
#Import-Module Microsoft.PowerShell.ThreadJob
Invoke-ARI -TenantID -Automation -StorageAccount "mystrageaccount" -StorageContainer "mystoragecontainer"
Runtime Environment PS 7.4
Az Azure CLI Az.Accounts Az.Compute Az.CostManagement Az.OperationalInsights Az.ResourceGraph Az.Storage AzureResourceInventory ImportExcel Microsoft.PowerShell.ThreadJob ThreadJob
Results
Runbook completely ran with errors. Got both .xlsx and .xml (for drawio) drawio shows all resources on all subscriptions but excel has only Overview, Subscriptions and Advisory sheet. Have tried with only Microsoft.PowerShell.ThreadJob imported, only ThreadJob imported and both imported in runtime environment but still got the same ThreadJob errors. Please advise.
Errors
5/7/2025, 8:13:04 AM - Warning: TenantId 'xxxxxxxxx-yyyy-zzzzz-abc-ddddddddddd' contains more than one active subscription. First one will be selected for further use. To select another subscription, use Set-AzContext.
5/7/2025, 8:13:04 AM - Warning: To override which subscription Connect-AzAccount selects by default, use Update-AzConfig -DefaultSubscriptionForLogin 00000000-0000-0000-0000-000000000000. Go to https://go.microsoft.com/fwlink/?linkid=2200610 for more information.
5/7/2025, 8:13:06 AM - Output:
Name : Default
Subscription : aaaaaaaa-bbbb-cccc-ddd-eeeeeeeeeeee
Account : MSI@50342
Environment : AzureCloud
Tenant : xxxxxxxxx-yyyy-zzzzz-abc-ddddddddddd
TokenCache :
VersionProfile :
ExtendedProperties : {}
5/7/2025, 8:13:45 AM - Output: Extraction Phase Finished 5/7/2025, 8:13:45 AM - Output: Total Extraction Time: 00:00:00:36:918 5/7/2025, 8:13:45 AM - Output: Invoking Draw.Io main function. 5/7/2025, 8:13:45 AM - Output: Id Name PSJobTypeName State HasMoreData Location Command
1 DiagramVariabl… ThreadJob NotStarted False PowerShell …
5/7/2025, 8:13:45 AM - Output: 2 Diagram_Subscr… ThreadJob NotStarted False PowerShell …
5/7/2025, 8:13:45 AM - Output: 3 Diagram_Organi… ThreadJob NotStarted False PowerShell …
5/7/2025, 8:13:46 AM - Output: 1 DiagramVariabl… ThreadJob Completed True PowerShell …
5/7/2025, 8:13:46 AM - Output: 4 Diagram_Networ… ThreadJob NotStarted False PowerShell …
5/7/2025, 8:14:18 AM - Output: 2 Diagram_Subscr… ThreadJob Completed True PowerShell …
5/7/2025, 8:14:18 AM - Output: 3 Diagram_Organi… ThreadJob Completed True PowerShell …
5/7/2025, 8:14:18 AM - Output: 4 Diagram_Networ… ThreadJob Completed True PowerShell …
5/7/2025, 8:14:18 AM - Output: [ [ [ "System.Xml.XmlElement" ] ] ] 5/7/2025, 8:14:19 AM - Output: [ [ [ "System.Xml.XmlElement" ] ], [ [ "System.Xml.XmlElement" ] ] ] 5/7/2025, 8:14:19 AM - Output: Starting Advisory Job 5/7/2025, 8:14:19 AM - Output: Starting Subscription Job 5/7/2025, 8:14:19 AM - Output: 2025-05-07_01_14_19 - Processing Resources in Automation Mode 5/7/2025, 8:14:21 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 7 and the name ResourceJob_AI because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 8 and the name ResourceJob_Analytics because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 9 and the name ResourceJob_APIs because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 10 and the name ResourceJob_Compute because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 11 and the name ResourceJob_Container because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 12 and the name ResourceJob_Database because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 13 and the name ResourceJob_Hybrid because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 14 and the name ResourceJob_Integration because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 15 and the name ResourceJob_IoT because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 16 and the name ResourceJob_Management because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 17 and the name ResourceJob_Monitoring because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 18 and the name ResourceJob_Network_1 because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 19 and the name ResourceJob_Network_2 because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:22 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 20 and the name ResourceJob_Security because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:23 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 21 and the name ResourceJob_Storage because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:23 AM - Error: Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 22 and the name ResourceJob_Web because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
5/7/2025, 8:14:26 AM - Output: Processing Phase Finished 5/7/2025, 8:14:26 AM - Output: Total Processing Time: 00:00:00:39:509 5/7/2025, 8:14:27 AM - Output: Starting to Build Excel Report. 5/7/2025, 8:22:37 AM - Warning: Could not find C:\AzureResourceInventory\ARI_Automation_Report_2025-05-07_01_13.xlsx 5/7/2025, 8:22:55 AM - Error: Close-ExcelPackage: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\3.ReportingFunctions\Start-ARIExcelExtraData.ps1:245 Line | 245 | Close-ExcelPackage $excel | ~~~~~~ | Cannot bind argument to parameter 'ExcelPackage' because it is null.
5/7/2025, 8:23:26 AM - Error: OperationStopped: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\3.ReportingFunctions\Start-ARIExtraReports.ps1:92 Line | 92 | … while (get-job -Name 'Advisory' | Where-Object { $_.State -eq 'R … | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | Exception of type 'System.OutOfMemoryException' was thrown.
5/7/2025, 8:23:27 AM - Error: Import-Module: Line | 3 | import-module $($args[1]) | ~~~~~~~~ | Exception of type 'System.OutOfMemoryException' was thrown.
5/7/2025, 8:23:35 AM - Error: Import-Module: Line | 3 | import-module $($args[2]) | ~~~~~~~~ | Exception of type 'System.OutOfMemoryException' was thrown.
5/7/2025, 8:23:43 AM - Warning: Failed adding PivotTable 'P3': Cannot index into a null array. 5/7/2025, 8:23:43 AM - Warning: Failed adding chart for pivotable 'P3': Cannot bind argument to parameter 'PivotTable' because it is null. 5/7/2025, 8:23:43 AM - Warning: Failed adding PivotTable 'P4': Cannot index into a null array. 5/7/2025, 8:23:43 AM - Warning: Failed adding chart for pivotable 'P4': Cannot bind argument to parameter 'PivotTable' because it is null. 5/7/2025, 8:23:43 AM - Warning: Failed adding PivotTable 'P5': Cannot index into a null array. 5/7/2025, 8:23:43 AM - Warning: Failed adding chart for pivotable 'P5': Cannot bind argument to parameter 'PivotTable' because it is null. 5/7/2025, 8:23:44 AM - Output: Report Building Finished 5/7/2025, 8:23:44 AM - Output: Total Processing Time: 00:00:09:17:827 5/7/2025, 8:23:46 AM - Output: Sending Excel file to Storage Account: 5/7/2025, 8:23:46 AM - Output: C:\AzureResourceInventory\ARI_Automation_Report_2025-05-07_01_13.xlsx 5/7/2025, 8:23:50 AM - Output: Sending Diagram file to Storage Account: 5/7/2025, 8:23:50 AM - Output: C:\AzureResourceInventory\ARI_Automation_Diagram_2025-05-07_01_13.xml
================================
Same problem here, also tried all modules - All modules are on the newest version, but I always get one of the following errors:
-
Start-ThreadJob: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Invoke-ARIAdvisoryJob.ps1:26 Line | 26 | Start-ThreadJob -Name 'Advisory' -ScriptBlock { | ~~~~~~~~~~~~~~~ | The 'Start-ThreadJob' command was found in the module 'ThreadJob', but the module could not be loaded. For more information, run 'Import-Module ThreadJob'.
-
Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 12 and the name ResourceJob_Database because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
Hello @kd-jessada
I believe this is more related with this error:
Exception of type 'System.OutOfMemoryException' was thrown.
How big is this environment? Is possible to run the inventory using Hybrid Workers?
Hi Claudio,
Currently I started to deploy ARI on sandbox tenant which has only 2 subscriptions and less than 500 resources. And I plan to deploy on production tenant which has more than 120 subscriptions and more than 20,000 resources if ARi on sandbox is successful.
I have successfully run ARI from my local machine. Sandbox took a few minutes to be done but on production tenant took almost 2 hours.
I also look at hybrid worker if I can’t solve this issue.
Same problem here, also tried all modules - All modules are on the newest version, but I always get one of the following errors:
- Start-ThreadJob: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Invoke-ARIAdvisoryJob.ps1:26 Line | 26 | Start-ThreadJob -Name 'Advisory' -ScriptBlock { | ~~~~~~~~~~~~~~~ | The 'Start-ThreadJob' command was found in the module 'ThreadJob', but the module could not be loaded. For more information, run 'Import-Module ThreadJob'.
- Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 12 and the name ResourceJob_Database because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')
Instead of ThreadJob, try the module Microsoft.Powershell.ThreadJob
Apparently the ThreadJob module was migrated to Microsoft.Powershell.ThreadJob
Hi Claudio,
Currently I started to deploy ARI on sandbox tenant which has only 2 subscriptions and less than 500 resources. And I plan to deploy on production tenant which has more than 120 subscriptions and more than 20,000 resources if ARi on sandbox is successful.
I have successfully run ARI from my local machine. Sandbox took a few minutes to be done but on production tenant took almost 2 hours.
I also look at hybrid worker if I can’t solve this issue.
Yea, something is up in the script, will take a deeper look and see if there is any bug in there
https://learn.microsoft.com/en-us/azure/automation/automation-runbook-execution#runbook-execution-environment
Task | Recommendation | Notes
Run a resource-intensive script | Hybrid Runbook Worker | Azure sandboxes have resource limits.
Hello,
I have the same problem. Also with only 4 subscriptions and DrawIO not in use. Hybrid Worker is not possible for me :-( My workaround is to install ARI 3.5.17.
Kind regards, Robert
I
Hello @kd-jessada
I believe this is more related with this error:
Exception of type 'System.OutOfMemoryException' was thrown.
How big is this environment? Is possible to run the inventory using Hybrid Workers?
On the Exception of type 'System.OutOfMemoryException' was thrown, I was receiving the same using an automation account and runbook and I'm now using -SkipDiagram which no longer throws the error.
@Claudio-Merola I'm having similar issues, using the automation guide with an automation account/runbook configured. Errors include:
Warning: "Could not find C:\AzureResourceInventory\ARI_Automation_Report_2025-05-09_14_52.xlsx"
Error: "Remove-Job: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\2.ProcessingFunctions\Build-ARICacheFiles.ps1:43 Line | 43 | Remove-Job -Name $Job | ~~~~~~~~~~~~~~~~~~~~~ | The command cannot remove the job with the job ID 3 and the name ResourceJob_AI because the job is not finished. To remove the job, first stop the job, or use the Force parameter. (Parameter 'Name')"
Error: Close-ExcelPackage: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\3.ReportingFunctions\Start-ARIExcelExtraData.ps1:245 Line | 245 | Close-ExcelPackage $excel | ~~~~~~ | Cannot bind argument to parameter 'ExcelPackage' because it is null.
Would you like me to raise a new issue? Thankyou
@TrueKillRob @plaaard @kd-jessada @GluRakH
I just published version 3.6.5.
Can you try this version and see if it fixed?
@Claudio-Merola I'm now using 3.6.5, getting errors: Exception of type 'System.OutOfMemoryException' was thrown.
ConvertFrom-Json: Line | 6 | $Resources = $($args[4]) | ConvertFrom-Json | ~~~~~~~~~~~~~~~~ | Object reference not set to an instance of an object.
Close-ExcelPackage: C:\usr\src\PSModules\AzureResourceInventory\modules\Private\3.ReportingFunctions\Start-ARIExcelExtraData.ps1:245 Line | 245 | Close-ExcelPackage $excel | ~~~~~~ | Cannot bind argument to parameter 'ExcelPackage' because it is null.
Warnings:
TenantId contains more than one active subscription. First one will be selected for further use. To select another subscription, use Set-AzContext.
Failed adding PivotTable 'P3': Cannot index into a null array.
Failed adding chart for pivotable 'P3': Cannot bind argument to parameter 'PivotTable' because it is null.
Failed adding PivotTable 'P4': Cannot index into a null array.
Failed adding chart for pivotable 'P4': Cannot bind argument to parameter 'PivotTable' because it is null.
Failed adding PivotTable 'P5': Cannot index into a null array.
Failed adding chart for pivotable 'P5': Cannot bind argument to parameter 'PivotTable' because it is null.
The .xlsx is now showing in the storage account, but when opened I'm only seeing three tabs "Overview", "Subscriptions" "Advisor" with all resources listed in "Subscriptions" and advisories in the "Advisor" tab.
@plaaard
the "Exception of type 'System.OutOfMemoryException'" means the jobs consumed too much memory of the Runtime environment.
I have changed the script to run batches of jobs of 4 at a time, to avoid OutOfMemoryException. How big is this environment? Can you test using Hybrid Workers?
Update the result of Automation on Windows Hybrid worker can run successfully without error.
Please be reminded to deploy with windows worker (not linux worker). Because ARI scripts call windows paths. When you try to run ARI on linux hybrid worker, it will be rejected.
Update the result of Automation on Windows Hybrid worker can run successfully without error.
Please be reminded to deploy with windows worker (not linux worker). Because ARI scripts call windows paths. When you try to run ARI on linux hybrid worker, it will be rejected.
Actually it should work on Linux too, do you have any error output from that Linux Worker? It would be interesting to add that as a future bugfix.
On Linux worker, job failed as soon as starting to run without log, error and job output. There’s only exception "This runbook type isn't supported by Linux hybrid workers"
the "Exception of type 'System.OutOfMemoryException'" means the jobs consumed too much memory of the Runtime environment.
I have changed the script to run batches of jobs of 4 at a time, to avoid OutOfMemoryException. How big is this environment? Can you test using Hybrid Workers?
We have 31 subscriptions, I did have this working before but I'm not sure what's happened.
@TrueKillRob @plaaard @kd-jessada @GluRakH
I just published version 3.6.5.
Can you try this version and see if it fixed?
Hi,
I tested it today with 3.6.5 and it still doesn't work. I get an out of memory error for every subscription. We are using Azure Automation and there we cannot change the RAM of the environment. Since we try to work with as few VMs as possible, we cannot use HybridWorker.
Sorry
I'm sorry, but I forgot to post my call:
Invoke-ARI -TenantID "$TenantID" -Automation -SkipDiagram -SkipAPIs -StorageAccount "$StorageAccountName" -StorageContainer "$ContainerName"
Of course the variables are filled ;-)
Best regards, Robert
On Linux worker, job failed as soon as starting to run without log, error and job output. There’s only exception "This runbook type isn't supported by Linux hybrid workers"
@kd-jessada This is very odd, I believe it should work, I haven't found anything in the docs about support limitations, May I ask you to open a new issue just for this?
@plaaard the "Exception of type 'System.OutOfMemoryException'" means the jobs consumed too much memory of the Runtime environment. I have changed the script to run batches of jobs of 4 at a time, to avoid OutOfMemoryException. How big is this environment? Can you test using Hybrid Workers?
We have 31 subscriptions, I did have this working before but I'm not sure what's happened.
@plaaard In the version 3.6.0 we drastically change the way the script process the resource jobs. Before we were splitting the resource list and sending each chunk of it to each jobs, this way the amount of memory used by the jobs were smaller. But the problem is that some resource modules require different resource types to search extra details. i.e: VMs need the complete list of NICs and Disks, to find the NICs used by the VM being reported. Since we were splitting the resource lists in different process the process cannot access the other process's memory space, as a consequence the report ended up incomplete, with some VMs missing the NIC data, disks, etc.
So now we are sending the complete list of resources to each job, which increases the memory space used by each process.
I'll see if I think of something to address this.
I'm sorry, but I forgot to post my call:
Invoke-ARI -TenantID "$TenantID" -Automation -SkipDiagram -SkipAPIs -StorageAccount "$StorageAccountName" -StorageContainer "$ContainerName"Of course the variables are filled ;-)
Best regards, Robert
@TrueKillRob Can you also add -SkipAdivosry -SkipPolicy?
Just to see if it helps
I'm sorry, but I forgot to post my call:
Invoke-ARI -TenantID "$TenantID" -Automation -SkipDiagram -SkipAPIs -StorageAccount "$StorageAccountName" -StorageContainer "$ContainerName"Of course the variables are filled ;-) Best regards, Robert@TrueKillRob Can you also add -SkipAdivosry -SkipPolicy?
Just to see if it helps
Sorry to say, but it does not help! Find the Output attached as TXT
@plaaard the "Exception of type 'System.OutOfMemoryException'" means the jobs consumed too much memory of the Runtime environment. I have changed the script to run batches of jobs of 4 at a time, to avoid OutOfMemoryException. How big is this environment? Can you test using Hybrid Workers?
We have 31 subscriptions, I did have this working before but I'm not sure what's happened.
@plaaard In the version 3.6.0 we drastically change the way the script process the resource jobs. Before we were splitting the resource list and sending each chunk of it to each jobs, this way the amount of memory used by the jobs were smaller. But the problem is that some resource modules require different resource types to search extra details. i.e: VMs need the complete list of NICs and Disks, to find the NICs used by the VM being reported. Since we were splitting the resource lists in different process the process cannot access the other process's memory space, as a consequence the report ended up incomplete, with some VMs missing the NIC data, disks, etc.
So now we are sending the complete list of resources to each job, which increases the memory space used by each process.
I'll see if I think of something to address this.
Thanks for the update, is it recommended to use 3.6.0 or can we go back to older versions?.
I'm sorry, but I forgot to post my call:
Invoke-ARI -TenantID "$TenantID" -Automation -SkipDiagram -SkipAPIs -StorageAccount "$StorageAccountName" -StorageContainer "$ContainerName"Of course the variables are filled ;-) Best regards, Robert@TrueKillRob Can you also add -SkipAdivosry -SkipPolicy? Just to see if it helps
Sorry to say, but it does not help! Find the Output attached as TXT
The odd part is that the change should not affect THAT much the memory consumed by the script.
The environment has resources in many different regions? I'm starting to suspect on the Quotas and SKU Details.
Can you instead try to use the parameter -SkipVMDetails?
@plaaard the "Exception of type 'System.OutOfMemoryException'" means the jobs consumed too much memory of the Runtime environment. I have changed the script to run batches of jobs of 4 at a time, to avoid OutOfMemoryException. How big is this environment? Can you test using Hybrid Workers?
We have 31 subscriptions, I did have this working before but I'm not sure what's happened.
@plaaard In the version 3.6.0 we drastically change the way the script process the resource jobs. Before we were splitting the resource list and sending each chunk of it to each jobs, this way the amount of memory used by the jobs were smaller. But the problem is that some resource modules require different resource types to search extra details. i.e: VMs need the complete list of NICs and Disks, to find the NICs used by the VM being reported. Since we were splitting the resource lists in different process the process cannot access the other process's memory space, as a consequence the report ended up incomplete, with some VMs missing the NIC data, disks, etc. So now we are sending the complete list of resources to each job, which increases the memory space used by each process. I'll see if I think of something to address this.
Thanks for the update, is it recommended to use 3.6.0 or can we go back to older versions?.
The newer version have more details and features than the older version. I would recommend the version 3.6.5+, but if it do not work in the environment the older version would be a better fit until we address the memory consumption issue in the current version.
I'm sorry, but I forgot to post my call:
Invoke-ARI -TenantID "$TenantID" -Automation -SkipDiagram -SkipAPIs -StorageAccount "$StorageAccountName" -StorageContainer "$ContainerName"Of course the variables are filled ;-) Best regards, Robert@TrueKillRob Can you also add -SkipAdivosry -SkipPolicy? Just to see if it helps
Sorry to say, but it does not help! Find the Output attached as TXT JobOutput.txt
The odd part is that the change should not affect THAT much the memory consumed by the script.
The environment has resources in many different regions? I'm starting to suspect on the Quotas and SKU Details.
Can you instead try to use the parameter -SkipVMDetails?
Great News: With SkipVMDetais there are only errors about some undefined DateTime variables. No more MemException! Btw: In your documentation is “SkipVMDetails” twice documented. What exactly do I lose with SkipVMDetails?
On a second run I added IncludeCosts and it also worked without MemException!
This is how I used it:
Invoke-ARI -TenantID "$TenantID" -Automation -SkipDiagram -SkipAPIs -SkipVMDetails -IncludeCosts -StorageAccount "$StorageAccountName" -StorageContainer "$ContainerName"
I'm sorry, but I forgot to post my call:
Invoke-ARI -TenantID "$TenantID" -Automation -SkipDiagram -SkipAPIs -StorageAccount "$StorageAccountName" -StorageContainer "$ContainerName"Of course the variables are filled ;-) Best regards, Robert@TrueKillRob Can you also add -SkipAdivosry -SkipPolicy? Just to see if it helps
Sorry to say, but it does not help! Find the Output attached as TXT JobOutput.txt
The odd part is that the change should not affect THAT much the memory consumed by the script. The environment has resources in many different regions? I'm starting to suspect on the Quotas and SKU Details. Can you instead try to use the parameter -SkipVMDetails?
Great News: With SkipVMDetais there are only errors about some undefined DateTime variables. No more MemException! Btw: In your documentation is “SkipVMDetails” twice documented. What exactly do I lose with SkipVMDetails?
On a second run I added IncludeCosts and it also worked without MemException!
This is how I used it:
Invoke-ARI -TenantID "$TenantID" -Automation -SkipDiagram -SkipAPIs -SkipVMDetails -IncludeCosts -StorageAccount "$StorageAccountName" -StorageContainer "$ContainerName"
Ok, that's a good news, I'll see if there are some improvements we can make in that.
You lose the Quota and VM Details for VM, AKS and VMSS:
@plaaard the "Exception of type 'System.OutOfMemoryException'" means the jobs consumed too much memory of the Runtime environment. I have changed the script to run batches of jobs of 4 at a time, to avoid OutOfMemoryException. How big is this environment? Can you test using Hybrid Workers?
We have 31 subscriptions, I did have this working before but I'm not sure what's happened.
@plaaard In the version 3.6.0 we drastically change the way the script process the resource jobs. Before we were splitting the resource list and sending each chunk of it to each jobs, this way the amount of memory used by the jobs were smaller. But the problem is that some resource modules require different resource types to search extra details. i.e: VMs need the complete list of NICs and Disks, to find the NICs used by the VM being reported. Since we were splitting the resource lists in different process the process cannot access the other process's memory space, as a consequence the report ended up incomplete, with some VMs missing the NIC data, disks, etc. So now we are sending the complete list of resources to each job, which increases the memory space used by each process. I'll see if I think of something to address this.
Thanks for the update, is it recommended to use 3.6.0 or can we go back to older versions?.
The newer version have more details and features than the older version. I would recommend the version 3.6.5+, but if it do not work in the environment the older version would be a better fit until we address the memory consumption issue in the current version.
Hello
I'm now using the below:
Azure Resource Inventory 3.6.5
With the below commands:
Invoke-ARI -TenantID -Automation -SkipDiagram -SkipAPIs -SkipVMDetails -IncludeCosts -StorageAccount -StorageContainer
This has created the ARI_Automation_Report and stored this into the storage account, I've given this a download and it looks better. It's created tabs including Overview, Subscriptions, Support Tickets, Advisors and for all the different resources we have in our tenant.
Runbook Job still reporting the below errors:
InvalidArgument: Line | 44 | $ProblemDate = [datetime]$ProblemDate | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | Cannot convert null to type "System.DateTime".
InvalidOperation: Line | 45 | … $ProblemDate = $ProblemDate.ToString("yyyy-MM-dd HH:mm") | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | You cannot call a method on a null-valued expression.
ConvertFrom-Json: Line | 6 | $Resources = $($args[4]) | ConvertFrom-Json | ~~~~~~~~~~~~~~~~ | Exception of type 'System.OutOfMemoryException' was thrown.
@plaaard
I already found the bug causing the: InvalidArgument: Line | 44 | $ProblemDate = [datetime]$ProblemDate | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | Cannot convert null to type "System.DateTime".
It will be fixed in the version 3.6.6.
The resource jobs are still failing due to 'System.OutOfMemoryException'. I'm working on some improvements regarding memory usage in version 3.6.6, and will also make the -Heavy parameter usable for Automation, I believe it should help to prevent the OutOfMemoryException.
@plaaard
I just published the version 3.6.6.
Can you try that version and see if the issue still occurs? Remember to add -Heavy in the parameters