PowerPlatform-DataverseServiceClient icon indicating copy to clipboard operation
PowerPlatform-DataverseServiceClient copied to clipboard

Unable to cast object of type 'generatedProxy_2' to type 'Microsoft.PowerPlatform.Dataverse.Client.IOrganizationServiceAsync'

Open sdhz236 opened this issue 2 years ago • 27 comments

I‘m getting this issue occasionally for azure function app, I'm using the latest verion of Microsoft.PowerPlatform.Dataverse.Client(0.6.1).

Full Exception :

Microsoft.PowerPlatform.Dataverse.Client.Utils.DataverseConnectionException : Failed to connect to Dataverse ---> System.AggregateException : One or more errors occurred. (Unable to cast object of type 'generatedProxy_2' to type 'Microsoft.PowerPlatform.Dataverse.Client.IOrganizationServiceAsync'.) ---> Unable to cast object of type 'generatedProxy_2' to type 'Microsoft.PowerPlatform.Dataverse.Client.IOrganizationServiceAsync'.

at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)

at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)

at System.Threading.Tasks.Task`1.get_Result()

at Microsoft.PowerPlatform.Dataverse.Client.ConnectionService.GetCachedService(ConnectionService& ConnectionObject)

at Microsoft.PowerPlatform.Dataverse.Client.ConnectionService.IntilizeService(ConnectionService& ConnectionObject)

at Microsoft.PowerPlatform.Dataverse.Client.ServiceClient.CreateServiceConnection(Object externalOrgServiceProxy,AuthenticationType requestedAuthType,String hostName,String port,String orgName,NetworkCredential credential,String userId,SecureString password,String domain,String Geo,String claimsHomeRealm,Boolean useSsl,Boolean useUniqueInstance,OrganizationDetail orgDetail,String clientId,Uri redirectUri,PromptBehavior promptBehavior,OrganizationWebProxyClientAsync externalOrgWebProxyClient,String certificateThumbPrint,StoreName certificateStoreName,X509Certificate2 certificate,Uri instanceUrl,Boolean isCloned,Boolean useDefaultCreds,Version incomingOrgVersion,ILogger externalLogger,String tokenCacheStorePath)

---> (Inner Exception #0) System.InvalidCastException : Unable to cast object of type 'generatedProxy_2' to type 'Microsoft.PowerPlatform.Dataverse.Client.IOrganizationServiceAsync'.

at async Microsoft.PowerPlatform.Dataverse.Client.ConnectionService.InitServiceAsync()<---

End of inner exception

at Microsoft.PowerPlatform.Dataverse.Client.ServiceClient.CreateServiceConnection(Object externalOrgServiceProxy,AuthenticationType requestedAuthType,String hostName,String port,String orgName,NetworkCredential credential,String userId,SecureString password,String domain,String Geo,String claimsHomeRealm,Boolean useSsl,Boolean useUniqueInstance,OrganizationDetail orgDetail,String clientId,Uri redirectUri,PromptBehavior promptBehavior,OrganizationWebProxyClientAsync externalOrgWebProxyClient,String certificateThumbPrint,StoreName certificateStoreName,X509Certificate2 certificate,Uri instanceUrl,Boolean isCloned,Boolean useDefaultCreds,Version incomingOrgVersion,ILogger externalLogger,String tokenCacheStorePath)

at Microsoft.PowerPlatform.Dataverse.Client.ServiceClient.ConnectToService(String connectionString,ILogger logger)

at Microsoft.PowerPlatform.Dataverse.Client.ServiceClient..ctor(String dataverseConnectionString,ILogger logger)

at myFunction.Startup.<>c.<Configure>b__2_0(IServiceProvider s) at D:\XXX\AzureFunction\AzureFunctions\myFunction\Startup.cs : 16

at DryIoc.Microsoft.DependencyInjection.DryIocAdapter.<>c__DisplayClass3_0.<RegisterDescriptor>b__0(IResolverContext r) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\DryIocAdapter.cs : 156

at DryIoc.Registrator.<>c__DisplayClass27_0.<RegisterDelegate>b__0(IResolverContext r) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 4550

at System.Linq.Expressions.Interpreter.FuncCallInstruction`3.Run(InterpretedFrame frame)

at System.Linq.Expressions.Interpreter.Interpreter.Run(InterpretedFrame frame)

at System.Linq.Expressions.Interpreter.LightLambda.Run(Object[] arguments)

at Thunk(Func`2 ,IResolverContext )

at DryIoc.Factory.<>c__DisplayClass26_0.<ApplyReuse>b__2() at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 6605

at DryIoc.Scope.TryGetOrAdd(ImMap`1 items,Int32 id,CreateScopedValue createValue,Int32 disposalOrder) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 7849

at DryIoc.Scope.GetOrAdd(Int32 id,CreateScopedValue createValue,Int32 disposalOrder) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 7834

at DryIoc.Factory.ApplyReuse(Expression serviceExpr,Request request) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 6604

at DryIoc.Factory.GetExpressionOrDefault(Request request) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 6564

at DryIoc.Factory.GetDelegateOrDefault(Request request) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 6634

at DryIoc.DelegateFactory.GetDelegateOrDefault(Request request) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 7740

at DryIoc.Container.ResolveAndCacheDefaultFactoryDelegate(Type serviceType,IfUnresolved ifUnresolved) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 211

at DryIoc.Container.DryIoc.IResolver.Resolve(Type serviceType,IfUnresolved ifUnresolved) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\DryIoc\Container.cs : 196

at Microsoft.Azure.WebJobs.Script.WebHost.DependencyInjection.ScopedServiceProvider.GetService(Type serviceType) at D:\a_work\1\s\src\WebJobs.Script.WebHost\DependencyInjection\ScopedServiceProvider.cs : 25

at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetService(IServiceProvider sp,Type type,Type requiredBy,Boolean isDefaultParameterRequired)

at lambda_method(Closure ,IServiceProvider ,Object[] )

at Microsoft.Azure.WebJobs.Host.Executors.DefaultJobActivator.CreateInstance[T](IServiceProvider serviceProvider) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\DefaultJobActivator.cs : 37

at Microsoft.Azure.WebJobs.Host.Executors.DefaultJobActivator.CreateInstance[T](IFunctionInstanceEx functionInstance) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\DefaultJobActivator.cs : 32

at Microsoft.Azure.WebJobs.Host.Executors.ActivatorInstanceFactory`1.<>c__DisplayClass1_1.<.ctor>b__0(IFunctionInstanceEx i) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\ActivatorInstanceFactory.cs : 20

at Microsoft.Azure.WebJobs.Host.Executors.ActivatorInstanceFactory`1.Create(IFunctionInstanceEx functionInstance) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\ActivatorInstanceFactory.cs : 26

at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.CreateInstance(IFunctionInstanceEx functionInstance) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs : 44

at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ParameterHelper.Initialize() at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 799

at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance,CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs : 104

sdhz236 avatar Apr 01 '22 08:04 sdhz236

We have encountered the same problem. There are a lot of errors. Do you have any progress on this problem?

MoranShusheng avatar Apr 13 '22 13:04 MoranShusheng

We have not been able to reliably duplicate the problem at this point and are speculating on what is going on. it would be extremely helpful if you could provide us a repro of this issue or more information about what is going on when it happens?

thanks MattB

MattB-msft avatar Apr 14 '22 00:04 MattB-msft

We have the same issue here. Sadly the project I'm working on is proprietary but I can reproduce the error in a reliable way:

Framework: .NET 6 Function Runtime Version: latest

Steps:

  1. Create a function app that has a startup function
  2. Initialize a ServiceClient with Client/Secret authentication within that startup function

This produces the mentioned error for us every time we start up the function app. Any pointers what the underlying issues could be? With the right pointers I could try to fix the issue myself and create a pull-request

Edit: We are working with the european tenants of Dynamics so our instances have the pattern <instancename>.crm4.dynamics.com

MortenRoemer avatar May 03 '22 15:05 MortenRoemer

The client does initialize? Or does it fail and you have a non working client?

MattB-msft avatar May 04 '22 03:05 MattB-msft

The Client fails to initialize. After that every call to Public members results in NullReferenceExceptions

MortenRoemer avatar May 04 '22 04:05 MortenRoemer

Can you paste your setup block here? I assume you have a startup.cs with a public override void Configure(IFunctionsHostBuilder builder) your adding the setup call too?

MattB-msft avatar May 04 '22 20:05 MattB-msft

As an example,... this is what one of our function test cases' startup.cs's looks like: This is Functions 4.0. ( this is not a formal example, but it is from an internal function we are using )

[assembly: FunctionsStartup(typeof(ExampleFunction_DI.Startup))]
namespace ExampleFunction_DI
{
    internal class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
            builder.Services.AddLogging();
            builder.Services.AddSingleton<ServiceClient>(sp =>
            {
                var logFactory = sp.GetService<ILoggerFactory>();
                var logger = logFactory.CreateLogger(typeof(ServiceClient));
                return GetClient(logger);
            });

        }

        static ServiceClient GetClient(ILogger logger = null)
        {
            logger?.LogInformation("Connecting to Dataverse");
            var connectionString = $"{Environment.GetEnvironmentVariable("ConnectionStringBase")};AppId={Environment.GetEnvironmentVariable("DataverseAppId")};ClientSecret={Environment.GetEnvironmentVariable("DataverseAppIdSecret")};Url={Environment.GetEnvironmentVariable("InstanceUri")}";
            ServiceClient client = null;
            client = new ServiceClient(connectionString, logger);
            if (client.IsReady)
            {
                logger?.LogInformation("Connected to Dataverse...");
                return client;
            }
            else
                throw client.LastException;
        }

    }
}

MattB-msft avatar May 05 '22 03:05 MattB-msft

Our startup function is a little bit simpler:

public class Startup : FunctionsStartup
{
    public override void ConfigureAppConfiguration(IFunctionsConfigurationBuilder builder)
    {
        builder.ConfigurationBuilder
            .AddUserSecrets(Assembly.GetExecutingAssembly(), true);

        base.ConfigureAppConfiguration(builder);
    }
    
    public override void Configure(IFunctionsHostBuilder builder)
    {
        LoadServices(builder.Services);
    }

    private static void LoadServices(IServiceCollection services)
    {
        services
            .AddSingleton<ServiceClient>(ConfigureDataverseClient);
    }

    private static ServiceClient ConfigureDataverseClient(IServiceProvider serviceProvider)
    {
        var client = new ServiceClient(new Uri(Config.DataverseUrl.Value!), Config.DataverseClientId.Value!, Config.DataverseClientSecret.Value!, true);
        return client;
    }
}

MortenRoemer avatar May 05 '22 06:05 MortenRoemer

EDIT: This workaround reduced the likelyhood of this issue but not prevented it. See post from 2022-05-09 for more info

But we found a workaround: When we handle the client/secret authentication ourselves it works without issues:

Here is our workaround startup function:

public class Startup : FunctionsStartup
{
    public override void ConfigureAppConfiguration(IFunctionsConfigurationBuilder builder)
    {
        builder.ConfigurationBuilder
            .AddUserSecrets(Assembly.GetExecutingAssembly(), true);

        base.ConfigureAppConfiguration(builder);
    }
    
    public override void Configure(IFunctionsHostBuilder builder)
    {
        LoadServices(builder.Services);
    }

    private static void LoadServices(IServiceCollection services)
    {
        services
            .AddSingleton<ServiceClient>(ConfigureDataverseClient);
    }

    private static ServiceClient ConfigureDataverseClient(IServiceProvider serviceProvider)
    {
        var httpClient = new HttpClient();
        var authHandler = new DataverseAuthHandler(
            httpClient, 
            Config.DataverseTenantId.Value!,
            Config.DataverseUrl.Value!,
            Config.DataverseClientId.Value!, 
            Config.DataverseClientSecret.Value!
        );
        var client = new ServiceClient(new Uri(Config.DataverseUrl.Value!), _ => authHandler.RetrieveToken(_));
        return client;
    }
}

And our helper class that handles the client/secret authentication:

public class DataverseAuthHandler
{
    public DataverseAuthHandler(HttpClient httpClient, string tenantId, string instanceUrl, string clientId, string clientSecret)
    {
        HttpClient = httpClient;
        LoginUrl = $"https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/token";
        InstanceUrl = instanceUrl;
        ClientId = clientId;
        ClientSecret = clientSecret;
    }
    
    private HttpClient HttpClient { get; }
    
    private string LoginUrl { get; }
    
    private string InstanceUrl { get; }
    
    private string ClientId { get; }
    
    private string ClientSecret { get; }
    
    private string? Token { get; set; }
    
    private DateTime? TokenValidUntil { get; set; }
    
    private bool IsTokenStillValid 
        => Token is not null && DateTime.UtcNow < TokenValidUntil!.Value;

    public async Task<string> RetrieveToken(string _)
    {
        if (IsTokenStillValid)
            return Token!;

        var tokenResult = await LogIn(InstanceUrl);
        Token = tokenResult.AccessToken;
        TokenValidUntil = DateTime.UtcNow + TimeSpan.FromMinutes(25);
        return Token;
    }

    private async Task<TokenResult> LogIn(string instanceUrl)
    {
        var scope = instanceUrl + ".default";
        var message = new HttpRequestMessage(HttpMethod.Post, LoginUrl)
        {
            Content = new FormUrlEncodedContent(
                new Dictionary<string, string>
                {
                    { "grant_type", "client_credentials" },
                    { "client_id", ClientId },
                    { "client_secret", ClientSecret },
                    { "scope", scope }
                }
            )
        };

        var response = await HttpClient.SendAsync(message);
        response.EnsureSuccessStatusCode();
        return JsonSerializer.Deserialize<TokenResult>(await response.Content.ReadAsStreamAsync())!;
    }

    private class TokenResult
    {
        [JsonPropertyName("token_type")]
        public string TokenType { get; set; } = null!;

        [JsonPropertyName("expires_in")]
        public int ExpiresIn { get; set; }

        [JsonPropertyName("ext_expires_in")]
        public int ExtExpiresIn { get; set; }

        [JsonPropertyName("access_token")]
        public string AccessToken { get; set; } = null!;
    }
}

MortenRoemer avatar May 05 '22 06:05 MortenRoemer

Are you using the 0.6.6 client? If not can you retest on the please .

MattB-msft avatar May 05 '22 13:05 MattB-msft

I've tested it initially on the 0.6.6 release. There I encountered the error

MortenRoemer avatar May 05 '22 13:05 MortenRoemer

Thanks, can you tell us about you hosting setup, What version of azure functions? What version of the azure functions library (nuget) and dependency injection your using? What OS host? Dedicated or shared sub for azure?

Thanks

MattB-msft avatar May 05 '22 13:05 MattB-msft

Here:

  • Azure Function Runtime: latest
  • Microsoft.NET.Sdk.Functions: 4.1.0
  • Microsoft.Extensions.Configuration 6.0.1
  • Host OS: Linux
  • dedicated subscription

Hope that helps

MortenRoemer avatar May 05 '22 13:05 MortenRoemer

We are, sofar, unable to duplicate this issue.

Can you add an ILogger to your function and capture logs for us of the failure?
To do that... Add the Ilogger ( Created As I showed above, the constructor your using will accept an ILogger as the final parameter ) and include in your host.json

...
  "logging": {
    "logLevel": {
      "Microsoft.PowerPlatform.Dataverse.Client.ServiceClient": "Trace"
    },
...

if you are not comfortable sharing the output here on Git, please go here: Nuget: Microsoft.PowerPlatform.Dataverse.Client and hit contact owner - middle of the page on the right and send an email referring to this issue

I will respond directly to you via email and you can reply with the log

Thanks for your help in helping us track this down.

MattB-msft avatar May 05 '22 15:05 MattB-msft

This may or may not help but wanted to confirm that I was also getting this issue, was using also using version 6.6, after seeing this and not wanting to try MortenRoemers clever workaround I tried going back down to 5.17 with same result.

Azure Function Runtime: latest Microsoft.NET.Sdk.Functions: 4.1.0 Host OS: Windows Consumption Plan .Net 6

Initially I was generating a new ServiceClient each time a request was required with a connection string.

Restarting the function would temporarily resolve the issue, the issue would then occur again after a period of time and then continue to happen, given it occurred in line with a spike in requests to the function I made an assumption (which i think is correct) that it occurs when the function auto scales out.

I implemented an easier but way stupiderer workaround which was that I have a declared a static ServiceClient and during startup I create the assign it.

Now, when I stare at the live logs on Azure I can see that this issue still occurs, but because it occurs during startup Microsoft helpfully abort this scale out attempt and makes a new instance (or maybe just abandons the scale out attempt entirely and restarts the service) which ends up working so when my functions needs it it doesn't bork out.

Either way, seeing as now I make the whole thing crap out when it tries to scale out the issue has to be something to do with reusing invalid info from another instance.

TomWinton avatar May 05 '22 17:05 TomWinton

Thats great info.. and may be the key to what is happening here.. Our tests were not generating enough load to scale out. Thanks @TomWinton,

MattB-msft avatar May 05 '22 21:05 MattB-msft

@TomWinton may be right that this is a scaling issue, so my workaround only reduced the error but not prevented it.

My project is sadly time-critical so we had to abandon this package in favor of a custom Web-API implementation. But after that project is finished I will gladly come back and provide you with the requested logs

MortenRoemer avatar May 09 '22 11:05 MortenRoemer

I think you can use Lazy pattern to avoid time cost of Create a ServiceClient, so you Startup will finish soon without be blocked by the constructor

8ggmaker avatar May 16 '22 03:05 8ggmaker

We are inflight on adding deliberate support for that feature set (delayed initialization) into the client itself. It should appear in the 1.0 release.

MattB-msft avatar May 16 '22 20:05 MattB-msft

Trying to consolidate messages here.

We are having a heck of a time replicating this issue in a controlled environment.
Our current load testing (simple commands) via functions is working though scale out just fine.
(Scaling out to 50 + processes without complaint.)

For folks seeing this issue, can you provide the following information:

  1. Version of DV Service Client your using
  2. Are you using AsyncMethods from the DV Service Client or sync Methods
  3. Are you using .Clone for your clients or using only a single instance or are you creating a connection each time?
  4. Using Onboard Auth or Custom Auth Hander
  5. Using DI or custom Cache
  6. If Using DI, are you using a Singleton or a scope
  7. What version of functions are you using? (version of Microsoft.NET.Sdk.Functions )
  8. What Function hosting plan are you using
  9. What Function Host type (Linux or Windows) are you using
  10. What level of concurrent calls are you seeing these failures on

Any information you could provide to help us chase this down would be greatly appreciated.

MattB-msft avatar Aug 05 '22 05:08 MattB-msft

Hi @MattB-msft

  1. Version of DV Service Client your using - 1.0.9
  2. Are you using AsyncMethods from the DV Service Client or sync Methods - AsyncMethods
  3. Are you using .Clone for your clients or using only a single instance or are you creating a connection each time? .Clone
  4. Using Onboard Auth or Custom Auth Hander - Onboard
  5. Using DI or custom Cache - DI
  6. If Using DI, are you using a Singleton or a scope - Scope
  7. What version of functions are you using? (version of Microsoft.NET.Sdk.Functions ) 4.1.0
  8. What Function hosting plan are you using - Consumption
  9. What Function Host type (Linux or Windows) are you using - Windows
  10. What level of concurrent calls are you seeing these failures on – When making a HttpTrigger Function request which wraps multiple inserts into different tables using ExecuteTransactionRequest and after the related records are created we have additional process in the background that utilizes Azure Durable Function to update existing records in Dynamics.

twiga2013 avatar Aug 05 '22 12:08 twiga2013

@twiga2013 Thanks for your info this really helps!. We were able to reproduce the problem with this configuration.

What we would like to you do is this, As your using Async Methods, do not use Clone(). (it's actually unnecessary if you're using Async methods) so, remove the using/clone() use and just use the singleton + Async methods.

Our tests are actually built that way ( use of execute async w/out clone ) which is why we have never seen this issue. We are debugging the WCF Client create issue (which is where we think this is)

MattB-msft avatar Aug 05 '22 17:08 MattB-msft

Hi @MattB-msft

The changes you recommend made a big difference. The Azure Functions were scaling out and was not see any errors. I will run some additional test under heavy load. Thank You so much for all your help.

twiga2013 avatar Aug 05 '22 19:08 twiga2013

@MattB-msft there's still locking going on when the SOAP endpoint is used. In that case you should use Clone()?

mathiasbl avatar Sep 21 '22 11:09 mathiasbl

@mathiasbl , when using the Async messages you should not be seeing locking on any endpoint, could you clarify what your seeing?

MattB-msft avatar Sep 21 '22 14:09 MattB-msft

I thought so by looking at the source code. An unsupported request, that uses soap, goes down to this method: https://github.com/microsoft/PowerPlatform-DataverseServiceClient/blob/dc278e33f6ad4e1b13335d70bb1ee53c6f8d9235/src/GeneralTools/DataverseClient/Client/ServiceClient.cs#L1677

mathiasbl avatar Sep 21 '22 14:09 mathiasbl

Correct, this should not lock the channel, are you seeing that happen?

MattB-msft avatar Sep 21 '22 14:09 MattB-msft

No, I misinterpreted the code.

I'm also seeing this issue in my functionapp (v4). Any progress on it? Fuction was running fine until this morning. This error happens on startup I think. Once started the function app will keep working.

My setup in DI:

 builder.Services.AddSingleton<TokenCredential>(sp => new DefaultAzureCredential());
            builder.Services.AddSingleton(sp =>
            {
                var tokenCredential = sp.GetService<TokenCredential>();

                var instanceUri = Environment.GetEnvironmentVariable("DataverseUri");
                var logFactory = sp.GetService<ILoggerFactory>();
                var logger = logFactory.CreateLogger(typeof(ServiceClient));

                var cache = sp.GetService<IMemoryCache>();

                return new ServiceClient(new Uri(instanceUri), async (url) =>
                {
                    var token = await cache.GetOrCreateAsync("token", async (item) =>
                    {
                        var scopes = new string[] { $"{instanceUri}/.default" };
                        var context = new TokenRequestContext(scopes: scopes);

                        var accesToken = await tokenCredential.GetTokenAsync(context, CancellationToken.None);

                        item.AbsoluteExpiration = accesToken.ExpiresOn.AddMinutes(-5);

                        return accesToken.Token;
                    });

                    return token;
                }, false, logger);
            });
  1. Version of DV Service Client your using - 1.0.23
  2. Are you using AsyncMethods from the DV Service Client or sync Methods - AsyncMethods
  3. Are you using .Clone for your clients or using only a single instance or are you creating a connection each time? No, single instance
  4. Using Onboard Auth or Custom Auth Hander - Custom
  5. Using DI or custom Cache - DI
  6. If Using DI, are you using a Singleton or a scope - Singleton
  7. What version of functions are you using? (version of Microsoft.NET.Sdk.Functions ) 4.1.0
  8. What Function hosting plan are you using - Consumption
  9. What Function Host type (Linux or Windows) are you using - Windows
  10. What level of concurrent calls are you seeing these failures on – 10 concurrent calls max through fan-out in durable function

Trace:

Unable to connect to Dataverse: Unable to cast object of type 'generatedProxy_2' to type 'Microsoft.PowerPlatform.Dataverse.Client.IOrganizationServiceAsync'.
Source: System.Private.CoreLib
Method: ChkCastAny
DateUTC: 11/17/2022
TimeUTC: 11:05:14 AM
Error: Unable to cast object of type 'generatedProxy_2' to type 'Microsoft.PowerPlatform.Dataverse.Client.IOrganizationServiceAsync'.
HelpLink Url: Not Provided
Stack Trace: at System.Reflection.DispatchProxy.Create[T,TProxy]()
   at System.ServiceModel.Channels.ServiceChannelProxy.CreateProxy[TChannel](MessageDirection direction, ServiceChannel serviceChannel)
   at System.ServiceModel.Channels.ServiceChannelFactory.CreateProxy[TChannel](MessageDirection direction, ServiceChannel serviceChannel)
   at System.ServiceModel.Channels.ServiceChannelFactory.CreateChannel[TChannel](EndpointAddress address, Uri via)
   at System.ServiceModel.ChannelFactory`1.CreateChannel(EndpointAddress address, Uri via)
   at System.ServiceModel.ChannelFactory`1.CreateChannel()
   at System.ServiceModel.ClientBase`1.CreateChannel()
   at System.ServiceModel.ClientBase`1.CreateChannelInternal()
   at System.ServiceModel.ClientBase`1.get_Channel()
   at System.ServiceModel.ClientBase`1.get_InnerChannel()
   at Microsoft.PowerPlatform.Dataverse.Client.ConnectionService.ConnectAndInitServiceAsync(OrganizationDetail orgdata, Boolean IsOnPrem, Uri homeRealmUri)
   at Microsoft.PowerPlatform.Dataverse.Client.ConnectionService.DoDirectLoginAsync(Boolean IsOnPrem)
   at Microsoft.PowerPlatform.Dataverse.Client.ConnectionService.InitServiceAsync()
======================================================================================================================

mathiasbl avatar Nov 17 '22 11:11 mathiasbl

@MattB-msft any progress? Still occasionally happens.

mathiasbl avatar Feb 06 '23 14:02 mathiasbl

We think this may be related to the memory leak issue here https://github.com/microsoft/PowerPlatform-DataverseServiceClient/issues/305

Our next update should resolve 305, please let us know after that release if this problem goes away for you

MattB-msft avatar Feb 14 '23 17:02 MattB-msft