ARM Unable to call service_principals.list on Azure::GraphRbac client

[jeffpereira]

I am trying to connect to the Graph API to get a list of service principals with the code below and have been unsuccessful.

tenant_id = <redacted>
client_id = <redacted>
app_key = <redacted>
subscription_id = <redacted>
provider = MsRestAzure::ApplicationTokenProvider.new(tenant_id, client_id, app_key)
credentials = MsRest::TokenCredentials.new(provider)
options = {
  tenant_id: tenant_id,
  client_id: client_id,
  client_secret: app_key,
  subscription_id: subscription_id,
  credentials: credentials
}

profile_client = Azure::GraphRbac::Profiles::Latest::Client.new(options)
profile_client.service_principals.client.tenant_id = tenant_id
profile_client.service_principals.list

I keep getting a 401 response like below:

MsRest::HttpOperationError: {
  "message": "MsRest::HttpOperationError",
  "request": {
    "base_uri": "https://graph.windows.net",
    "path_template": "{tenantID}/servicePrincipals",
    "method": "get",
    "path_params": {
      "tenantID": "<redacted>"
    },
    "skip_encoding_path_params": null,
    "query_params": {
      "$filter": null,
      "api-version": "1.6"
    },
    "skip_encoding_query_params": null,
    "headers": {
      "Content-Type": "application/json; charset=utf-8",
      "Accept": "application/json",
      "accept-language": "en-US",
      "x-ms-client-request-id": "<redacted>"
    },
    "body": null,
    "middlewares": [
      [
        "MsRest::RetryPolicyMiddleware",
        {
          "times": 3,
          "retry": 0.02
        }
      ],
      [
        "cookie_jar"
      ]
    ],
    "log": null
  },
  "response": {
    "body": "{\"odata.error\":{\"code\":\"Authentication_MissingOrMalformed\",\"message\":{\"lang\":\"en\",\"value\":\"Access Token missing or malformed.\"},\"date\":\"2018-02-14T19:47:57\",\"requestId\":\"<redacted>\",\"values\":null}}",
    "headers": {
      "cache-control": "private",
      "content-type": "application/json; odata=minimalmetadata; charset=utf-8",
      "server": "Microsoft-IIS/8.5",
      "ocp-aad-diagnostics-server-name": "<redacted>",
      "request-id": "<redacted>",
      "client-request-id": "<redacted>",
      "x-ms-dirapi-data-contract-version": "1.6",
      "strict-transport-security": "max-age=31536000; includeSubDomains",
      "access-control-allow-origin": "*",
      "x-aspnet-version": "4.0.30319",
      "x-powered-by": "ASP.NET, ASP.NET",
      "duration": "216396",
      "date": "Wed, 14 Feb 2018 19:47:54 GMT",
      "connection": "close",
      "content-length": "223"
    },
    "status": 401
  }
}

Using a debugger I was able to see the header being set properly in gems/ms_rest-0.7.2/lib/ms_rest/credentials/token_credentials.rb , but at the time of the request it does not seem to be there.

I also wanted to not that the credentials and IDs being used to authenticate work fine with other profiles. e.g. Azure::Authorization::Profiles::Latest::Mgmt::Client

[sarangan12]

@jeffpereira Thanks for reporting this issue. This seems like a service related issue. I have contacted the respective team and working with them in resolving this issue. I will update the issue once I have more details.

@salameer FYI.....

@darshanhs90 @danieldobalian @jmprieur Please look into this issue.

[jeffpereira]

As an update to this ticket, I tried inserting the authentication header (with JWT) in manually as a custom header, and it was still removed before the request .

[jeffpereira]

@sarangan12 Another update on this ticket. Aside from the token being stripped out there is an additional underlying issue.

I ran into a auth problem while using the same token generation mechanism for calling the Azure public API, and dug further into the issue. The token is not being generated with the proper audience to authenticate against any calls to graph.windows.net. While digging into the SDK a bit, it looks like the SDK technically has the ability to create the correct token with the correct audience, but there is no easy way to this without hacking around the SDK a bit. I was able to create a proper token with the SDK and use the API to get a valid response.

class AzureClient
  require 'net/http'
  attr_reader :tenant_id, :client_id, :app_key, :subscription_id, :graph_provider, :graph_credentials

  def initialize(tenant_id, client_id, app_key, subscription_id)
    @tenant_id = tenant_id
    @client_id = client_id
    @app_key = app_key
    @subscription_id = subscription_id
    @graph_provider = MsRestAzure::ApplicationTokenProvider.new(tenant_id, client_id, app_key, graph_provider_settings)
    @graph_credentials = MsRest::TokenCredentials.new(graph_provider)
    @service_principal = service_principal
  end

  def service_principal
    JSON.parse(service_principal_list.body)
  end

  private

  def graph_provider_settings
    azure_environment = MsRestAzure::AzureEnvironments::Azure
    settings = MsRestAzure::ActiveDirectoryServiceSettings.new
    settings.authentication_endpoint = azure_environment.active_directory_endpoint_url
    settings.token_audience = azure_environment.active_directory_graph_resource_id
    settings
  end

  def service_principal_list
    uri = URI("https://graph.windows.net/#{tenant_id}/servicePrincipals?api-version=1.6")

    Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https') do |http|
      request = Net::HTTP::Get.new uri
      request['Authorization'] = graph_provider.get_authentication_header

      http.request request # Net::HTTPResponse object
    end
  end
end

Using the above code with valid data for tenant_id, client_id, app_key, and subscription_id I was able to receive a valid response from the API. I do not believe that the service is the issue in this case since I am able to get a response just fine with valid data. I think that the two issue are that the SDK is stripping out the auth header prior to the request, and secondly, that there is no simple way to create the correct type of token for connecting to the various services with different hosts.

[jeffpereira]

Any status updates on this? It has been over a month.

[jeffpereira]

has a fix for this been deployed yet?

[cryonex]

@jeffpereira I just ran into this same issue today. I was able to get it working based on your previous example. You're correct that it doesn't use the correct token audience and auth endpoint by default (which is definitely a flaw IMO). In the current version, the correct auth headers seem to be preserved. Maybe that was fixed sometime in the past 10 months.

Here is what worked for me:

#!/usr/bin/env ruby

require 'azure_graph_rbac'

azure_acct_config = {
  tenant_id: YOUR_TENANT_ID,
  client_id: YOUR_CLIENT_ID,
  client_secret: YOUR_CLIENT_SECRET,
  subscription_id: YOUR_SUBSCRIPTION_ID
}
active_directory_settings = MsRestAzure::ActiveDirectoryServiceSettings.new
active_directory_settings.authentication_endpoint = MsRestAzure::AzureEnvironments::AzureCloud.active_directory_endpoint_url
active_directory_settings.token_audience = MsRestAzure::AzureEnvironments::AzureCloud.active_directory_graph_resource_id
rbac_client = Azure::GraphRbac::Profiles::Latest::Client.new(azure_acct_config.merge(active_directory_settings: active_directory_settings))
p rbac_client.service_principals.list

[viral1701]

Hi @cryonex Using your above example I have an issue where I keep getting /rubies/ruby-2.5.1/lib/ruby/gems/2.5.0/gems/azure_graph_rbac-0.17.0/lib/1.6/generated/azure_graph_rbac/service_principals.rb:160:in list_async': @client.tenant_id is nil (ArgumentError)

Have you seen this issue before?

[cryonex]

@viral1981 Oh right, I forgot that I patched the gem locally as well. You'll need to patch gems/azure_graph_rbac-0.17.0/lib/profiles/latest/modules/graphrbac_profile_module.rb with the following:

--- graphrbac_profile_module_old.rb	2018-12-12 14:17:38.226637535 -0700
+++ graphrbac_profile_module.rb	2018-12-11 13:26:04.063249620 -0700
@@ -80,6 +80,9 @@
       if(@client_0.respond_to?(:subscription_id))
         @client_0.subscription_id = configurable.subscription_id
       end
+      if(@client_0.respond_to?(:tenant_id))
+        @client_0.tenant_id = configurable.tenant_id
+      end
       add_telemetry(@client_0)
       @objects = @client_0.objects
       @applications = @client_0.applications

[stefangordon]

@sarangan12 is this really still non-functional without editing the graphrbac module? Is there some other way to use this client that I am missing?

Fwiw seems like you can also hack around this by doing this:

rbac_client.instance_variable_get(:@client_0).tenant_id = tenant_id
result = rbac_client.service_principals.list

[kurtzeborn]

Thank you for your interest in Azure SDKs. As detailed in this retirement announcement, this repo is no longer supported as of December 31st 2021. Please find the up-to-date list of languages and services supported with Azure SDKs here: https://aka.ms/azsdk