invoke.py grpc hanging on azure role definition requests with bad role names

[AMEvers]

What happened?

When using pulumi Azure python sdk, I am noticing that pulumi is hanging on calls to pulumi_azure.authentication.get_role_definition() when provided a bad role definition name. Correct names return a valid role definition and providing the role_definition_id works as expected with a bad role definition id raising an exception.

I tracked the issue to invoke.py with monitor.Invoke(req) in do_invoke() never returning a response/error. In my hanging case the ResourceInvokeRequest looks like

tok: "azure:authorization/getRoleDefinition:getRoleDefinition"
args {
  fields {
    key: "scope"
    value {
      string_value: "/"
    }
  }
  fields {
    key: "name"
    value {
      string_value: "test"
    }
  }
}
acceptResources: true

where 'test' is an invalid role definition name. I've let the call run for an hour without the hang ever resolving. This is not an azure auth issue. My code is able to create/manage other azure objects and, when provided a valid role name, the correct role definition is being returned. I would not be surprised if this is an issue with the Azure grpc server and pulumi is forever awaiting a response but I would expect that pulumi would have a timeout to prevent it from hanging indefinitely (or longer than my patience allows) on a grpc call.

The resultant logs from a bad up call:

Previewing update (): pulumi:pulumi:Stack: (same) [urn=urn:pulumi:::::pulumi:pulumi:Stack::] @ previewing update.... @ previewing update.... @ previewing update.... @ previewing update.... @ previewing update.... @ previewing update.... etc

Expected Behavior

invoke.py should be throwing an exception instead of hanging.

Steps to reproduce

Call pulumi_azure.authorization.get_role_definition(name='') If it matters at all, this was done via an in-line pulumi function.

Output of pulumi about

CLI
Version 3.72.2 Go Version go1.20.5 Go Compiler gc

Plugins NAME VERSION azure 5.43.0 azure-native 1.103.0 azuread 5.38.0 python unknown

Host
OS ubuntu Version 20.04 Arch x86_64

This project is written in python: executable='/usr/bin/python3' version='3.8.10 '

Pulumi locates its logs in /tmp by default warning: Failed to get information about the Pulumi program's dependencies: illegal semver returned by language host: [email protected]: Invalid character(s) found in patch number "0b1" warning: Failed to get information about the current stack: No current stack

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[dixler]

Hi, taking a look.

Was able to repro on pulumi-azure with

from pulumi_azure import authorization

name = "invalid_role_name"
role_def = authorization.get_role_definition(name)

[dixler]

Hi, after some looking, I'm going to move this to pulumi-azure and they may be able to better address this.

[thomas11]

To narrow down the problem, I tested with a TypeScript program and it behaves the same.

Turning on verbose logging, I see that the request to Azure is https://management.azure.com//providers/Microsoft.Authorization/roleDefinitions?%!f(MISSING)ilter=roleName+eq+%!i(MISSING)nvalid%!&(MISSING)api-version=2018-01-01-preview and Azure responds with 200 and a body of {[]}. Apart from the MISSING which are probably a logging artifact, this looks correct. Maybe we wrongly interpret an empty result as an error condition somewhere?