Enterprise gateway will not refresh access token. Custom Connector is using Aad for Authentication Kind

[dbeavon]

I have a custom connector that relies on the authentication kind "AAD" (Azure identities). The custom connector internally relies on a remote REST service. After an hour my AAD access token expires, and the mashup engine continues to send requests to the remote service with the old, expired access token. This results in failure. There is only one mashup container running after the hour has passed, when the failure occurs (ie. this is not likely to be a concurrency-related issue).

My custom power query (PQ) logic is long-running and is internally making use of the custom connector to call the Web.Contents interface. The REST calls which are initiated are brief (30 seconds or less) however there are lots of them which are sent in a loop (~200 or so). After a little over an hour the PQ fails because of a bad access token. The mashup engine refuses to refresh the token for some reason, even though the remote REST service is returning a 401.

I had spent some time investigating the behavior of the mashup engine itself (Microsoft.Mashup.Container.NetFX45.exe) and it seems to recognize that the token has expired, and it seems to want to refresh the token. See my research here: https://github.com/microsoft/DataConnectors/issues/494

... however, I don't think the mashup engine itself is responsible for generating a new token. It appears to delegate that out of process (possibly to Microsoft.PowerBI.EnterpriseGateway.exe?).

I believe this could be a known bug in the gateway, and could be one that has been present for a long time (see the following that seems to refer to it as well: https://community.powerbi.com/t5/Developer/Custom-data-connector-using-oauth2-not-refreshing-access-token/td-p/1275418 )

Our custom connector has become a critical component of our BI infrastructure and we would love to get a fix for this problem. I had thought we were using custom connectors in a pretty mainstream way, especially given that the authentication kind is AAD.