azure-cli-extensions icon indicating copy to clipboard operation
azure-cli-extensions copied to clipboard

We are not able to get "extensionSettings" values by using az monitor data-collection rule show in monitor-control-service 1.0.1.

Open m-yuka746 opened this issue 1 year ago • 4 comments

Describe the bug

Issue We are not able to get "extensionSettings" values by using az monitor data-collection rule show in monitor-control-service 1.0.1. Using previous version such as 0.4.1, 0.3.1, we are able to get "extensionSettings".

In addition, using Get-AzDataCollectionRule in Az.Monitor module, we are able to get "extensionSettings". https://www.powershellgallery.com/packages/Az.Monitor/5.0.0

Question Is it design changed from monitor-control-service 1.0.1 or bug for az monitor data-collection rule show? If bug, will the design be changed next release?

Thanks.

Related command

az monitor data-collection rule show

Errors

No error

Issue script & Debug output

using monitor-control-service 1.0.1, extensionSettings is null. az monitor data-collection rule show --resource-group 'defaultresourcegroup-ejp' --name 'ct-dcr-1622755568'

{
  "dataFlows": [
    {
      "destinations": [
        "Microsoft-CT-Dest"
      ],
      "streams": [
        "Microsoft-ConfigurationChange",
        "Microsoft-ConfigurationChangeV2",
        "Microsoft-ConfigurationData"
      ]
    }
  ],
  "dataSources": {
    "extensions": [
      {
        "extensionName": "ChangeTracking-Windows",
        **"extensionSettings": {},**
        "name": "CTDataSource-Windows",
        "streams": [
          "Microsoft-ConfigurationChange",
          "Microsoft-ConfigurationChangeV2",
          "Microsoft-ConfigurationData"
        ]
      },
      {
        "extensionName": "ChangeTracking-Linux",
        **"extensionSettings": {},**
        "name": "CTDataSource-Linux",
        "streams": [
          "Microsoft-ConfigurationChange",
          "Microsoft-ConfigurationChangeV2",
          "Microsoft-ConfigurationData"
        ]
      }
    ]
  },
...
  "type": "Microsoft.Insights/dataCollectionRules"
}

Expected behavior

using monitor-control-service 0.4.1, extensionSettings is not null.

az --version

azure-cli                         2.55.0
core                              2.55.0
telemetry                          1.1.0
Extensions:
ai-examples                        0.2.5
connectedk8s                       1.5.4
ml                                2.22.0
monitor-control-service               0.4.1
ssh                                2.0.2
Dependencies:
msal                            1.24.0b2
azure-mgmt-resource             23.1.0b2
Python location '/usr/bin/python3.9'
Extensions directory '/home/yuka/.azure/cliextensions'
Extensions system directory '/usr/lib/python3.9/site-packages/azure-cli-extensions'
Python (Linux) 3.9.14 (main, Oct 12 2023, 19:48:32) 
[GCC 11.2.0]
Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

az monitor data-collection rule show --resource-group 'defaultresourcegroup-ejp' --name 'ct-dcr-1622755568'

{           
  "dataCollectionEndpointId": null,
  "dataFlows": [
    {
      "destinations": [
        "Microsoft-CT-Dest"
      ],
      "outputStream": null,
      "streams": [
        "Microsoft-ConfigurationChange",
        "Microsoft-ConfigurationChangeV2",
        "Microsoft-ConfigurationData"
      ],
      "transformKql": null
    }
  ],
  "dataSources": {
    "extensions": [
      {
        "extensionName": "ChangeTracking-Windows",
        **"extensionSettings": {**
          "enableFiles": true,
          "enableInventory": true,
          "enableRegistry": true,
          "enableServices": true,
          "enableSoftware": true,
          "fileSettings": {
            "fileCollectionFrequency": 2700,
            "fileinfo": [
              {
                "checksum": "Md5",
                "description": "",
                "enabled": true,
                "groupTag": "Custom",
                "maxContentsReturnable": 5000000,
                "maxOutputSize": 0,
                "name": "custom log files",
                "path": "C:\\AMACustomLogs\\*.log",
                "pathType": "File",
                "recurse": true,
                "uploadContent": false
              }
            ]
          },
          "inventorySettings": {
            "inventoryCollectionFrequency": 36000
          },
          "registrySettings": {
            "registryCollectionFrequency": 3000,
            "registryInfo": [
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Group Policy\\Scripts\\Startup",
                "name": "Registry_1",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Group Policy\\Scripts\\Shutdown",
                "name": "Registry_2",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\SOFTWARE\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Run",
                "name": "Registry_3",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Active Setup\\Installed Components",
                "name": "Registry_4",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Classes\\Directory\\ShellEx\\ContextMenuHandlers",
                "name": "Registry_5",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Classes\\Directory\\Background\\ShellEx\\ContextMenuHandlers",
                "name": "Registry_6",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Classes\\Directory\\Shellex\\CopyHookHandlers",
                "name": "Registry_7",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\ShellIconOverlayIdentifiers",
                "name": "Registry_8",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Explorer\\ShellIconOverlayIdentifiers",
                "name": "Registry_9",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Browser Helper Objects",
                "name": "Registry_10",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Browser Helper Objects",
                "name": "Registry_11",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Internet Explorer\\Extensions",
                "name": "Registry_12",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Wow6432Node\\Microsoft\\Internet Explorer\\Extensions",
                "name": "Registry_13",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows NT\\CurrentVersion\\Drivers32",
                "name": "Registry_14",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\Software\\Wow6432Node\\Microsoft\\Windows NT\\CurrentVersion\\Drivers32",
                "name": "Registry_15",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\System\\CurrentControlSet\\Control\\Session Manager\\KnownDlls",
                "name": "Registry_16",
                "recurse": true,
                "valueName": ""
              },
              {
                "description": "",
                "enabled": false,
                "groupTag": "Recommended",
                "keyName": "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon\\Notify",
                "name": "Registry_17",
                "recurse": true,
                "valueName": ""
              }
            ]
          },
          "servicesSettings": {
            "serviceCollectionFrequency": 1800
          },
          "softwareSettings": {
            "softwareCollectionFrequency": 1800
          }
        },
        "inputDataSources": null,
        "name": "CTDataSource-Windows",
        "streams": [
          "Microsoft-ConfigurationChange",
          "Microsoft-ConfigurationChangeV2",
          "Microsoft-ConfigurationData"
        ]
      },
      {
        "extensionName": "ChangeTracking-Linux",
        "extensionSettings": {
          "enableFiles": true,
          "enableInventory": true,
          "enableRegistry": false,
          "enableServices": true,
          "enableSoftware": true,
          "fileSettings": {
            "fileCollectionFrequency": 900,
            "fileInfo": [
              {
                "destinationPath": "/etc/.*.conf",
                "enabled": true,
                "groupTag": "Recommended",
                "links": "Follow",
                "maxContentsReturnable": 5000000,
                "maxOutputSize": 500000,
                "name": "ChangeTrackingLinuxPath_default",
                "pathType": "File",
                "recurse": true,
                "type": "File",
                "useSudo": true
              }
            ]
          },
          "inventorySettings": {
            "inventoryCollectionFrequency": 36000
          },
          "servicesSettings": {
            "serviceCollectionFrequency": 300
          },
          "softwareSettings": {
            "softwareCollectionFrequency": 300
          }
        },
        "inputDataSources": null,
        "name": "CTDataSource-Linux",
        "streams": [
          "Microsoft-ConfigurationChange",
          "Microsoft-ConfigurationChangeV2",
          "Microsoft-ConfigurationData"
        ]
      }
    ],
    "iisLogs": null,
    "logFiles": null,
    "performanceCounters": null,
    "syslog": null,
    "windowsEventLogs": null
  },
 ...
  "type": "Microsoft.Insights/dataCollectionRules"
}

Environment Summary

az --version

azure-cli                         2.55.0
core                              2.55.0
telemetry                          1.1.0
Extensions:
ai-examples                        0.2.5
connectedk8s                       1.5.4
ml                                2.22.0
monitor-control-service               1.0.1
ssh                                2.0.2
Dependencies:
msal                            1.24.0b2
azure-mgmt-resource             23.1.0b2
Python location '/usr/bin/python3.9'
Extensions directory '/home/yuka/.azure/cliextensions'
Extensions system directory '/usr/lib/python3.9/site-packages/azure-cli-extensions'
Python (Linux) 3.9.14 (main, Oct 12 2023, 19:48:32) 
[GCC 11.2.0]
Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

Using Get-AzDataCollectionRule, we are able to get DataSourceExtension.

Get-Module -Name Az.Monitor -ListAvailable | Select-Object -Property Name,Version

Name       Version
----       -------
Az.Monitor 5.0.0

Get-AzDataCollectionRule -ResourceGroupName 'defaultresourcegroup-ejp' -Name 'ct-dcr-1622755568'

DataCollectionEndpointId                  : 
DataFlow                                  : {{
                                              "streams": [ "Microsoft-ConfigurationChange", "Microsoft-ConfigurationChangeV2", "Microsoft-ConfigurationData" ],
                                              "destinations": [ "Microsoft-CT-Dest" ]
                                            }}
DataSourceDataImportEventHubConsumerGroup : 
DataSourceDataImportEventHubName          : 
DataSourceDataImportEventHubStream        : 
DataSourceExtension                       : {{
                                              "streams": [ "Microsoft-ConfigurationChange", "Microsoft-ConfigurationChangeV2", "Microsoft-ConfigurationData" ],
                                              "extensionName": "ChangeTracking-Windows",
                                              "extensionSettings": {
                                                "enableFiles": true,
                                                "enableSoftware": true,
                                                "enableRegistry": true,
                                                "enableServices": true,
                                                "enableInventory": true,
                                                "registrySettings": {
                                                  "registryCollectionFrequency": 3000,
                                                  "registryInfo": [
                                                    {
                                                      "name": "Registry_1",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Group Policy\\Scripts\\Startup",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_2",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Group Policy\\Scripts\\Shutdown",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_3",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\SOFTWARE\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Run",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_4",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Active Setup\\Installed Components",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_5",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Classes\\Directory\\ShellEx\\ContextMenuHandlers",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_6",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Classes\\Directory\\Background\\ShellEx\\ContextMenuHandlers",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_7",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Classes\\Directory\\Shellex\\CopyHookHandlers",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_8",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": 
                                            "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\ShellIconOverlayIdentifiers",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_9",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": 
                                            "HKEY_LOCAL_MACHINE\\Software\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Explorer\\ShellIconOverlayIdentifiers",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_10",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Browser Helper 
                                            Objects",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_11",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Browser 
                                            Helper Objects",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_12",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Internet Explorer\\Extensions",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_13",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Wow6432Node\\Microsoft\\Internet Explorer\\Extensions",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_14",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows NT\\CurrentVersion\\Drivers32",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_15",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\Software\\Wow6432Node\\Microsoft\\Windows NT\\CurrentVersion\\Drivers32",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_16",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\System\\CurrentControlSet\\Control\\Session Manager\\KnownDlls",
                                                      "valueName": ""
                                                    },
                                                    {
                                                      "name": "Registry_17",
                                                      "groupTag": "Recommended",
                                                      "enabled": false,
                                                      "recurse": true,
                                                      "description": "",
                                                      "keyName": "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon\\Notify",
                                                      "valueName": ""
                                                    }
                                                  ]
                                                },
                                                "fileSettings": {
                                                  "fileCollectionFrequency": 2700,
                                                  "fileinfo": [
                                                    {
                                                      "name": "custom log files",
                                                      "enabled": true,
                                                      "description": "",
                                                      "path": "C:\\AMACustomLogs\\*.log",
                                                      "recurse": true,
                                                      "uploadContent": false,
                                                      "maxContentsReturnable": 5000000,
                                                      "maxOutputSize": 0,
                                                      "checksum": "Md5",
                                                      "pathType": "File",
                                                      "groupTag": "Custom"
                                                    }
                                                  ]
                                                },
                                                "softwareSettings": {
                                                  "softwareCollectionFrequency": 1800
                                                },
                                                "inventorySettings": {
                                                  "inventoryCollectionFrequency": 36000
                                                },
                                                "servicesSettings": {
                                                  "serviceCollectionFrequency": 1800
                                                }
                                              },
                                              "name": "CTDataSource-Windows"
                                            }, {
                                              "streams": [ "Microsoft-ConfigurationChange", "Microsoft-ConfigurationChangeV2", "Microsoft-ConfigurationData" ],
                                              "extensionName": "ChangeTracking-Linux",
                                              "extensionSettings": {
                                                "enableFiles": true,
                                                "enableSoftware": true,
                                                "enableRegistry": false,
                                                "enableServices": true,
                                                "enableInventory": true,
                                                "fileSettings": {
                                                  "fileCollectionFrequency": 900,
                                                  "fileInfo": [
                                                    {
                                                      "name": "ChangeTrackingLinuxPath_default",
                                                      "enabled": true,
                                                      "destinationPath": "/etc/.*.conf",
                                                      "useSudo": true,
                                                      "recurse": true,
                                                      "maxContentsReturnable": 5000000,
                                                      "pathType": "File",
                                                      "type": "File",
                                                      "links": "Follow",
                                                      "maxOutputSize": 500000,
                                                      "groupTag": "Recommended"
                                                    }
                                                  ]
                                                },
                                                "softwareSettings": {
                                                  "softwareCollectionFrequency": 300
                                                },
                                                "inventorySettings": {
                                                  "inventoryCollectionFrequency": 36000
                                                },
                                                "servicesSettings": {
                                                  "serviceCollectionFrequency": 300
                                                }
                                              },
                                              "name": "CTDataSource-Linux"
                                            }}
DataSourceIisLog                          : 
DataSourceLogFile                         : 
DataSourcePerformanceCounter              : 
DataSourcePlatformTelemetry               : 
DataSourcePrometheusForwarder             : 
DataSourceSyslog                          : 
DataSourceWindowsEventLog                 : 
DataSourceWindowsFirewallLog              : 
...
Type                                      : Microsoft.Insights/dataCollectionRules

m-yuka746 avatar Jan 07 '24 02:01 m-yuka746