Remoting V2_1 throws SerializationException

[vskh]

I am experiencing the issue when trying to use SF remoting v2_1, as explained here.

I've created a sample minimal project to illustrate the issue: https://github.com/vskh/sf-remoting-serialization-issue. It is mostly a default SF application with a single stateless service which exposes remoting listener via (non-IService) interface and tries to consume itself.

If I run it, I'm getting exception when attempting to call the method via remoting proxy: Type 'Microsoft.ServiceFabric.Services.Remoting.V2.ServiceRemotingResponseMessageBody' with data contract name 'msgResponse:urn:ServiceFabric.Communication' is not expected. Consider using a DataContractResolver if you are using DataContractSerializer or add any types not known statically to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding them to the list of known types passed to the serializer. Stacktrace:

System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeAndVerifyType(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, Boolean verifyKnownType, RuntimeTypeHandle declaredTypeHandle, Type declaredType)
   at System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithXsiTypeAtTopLevel(DataContract dataContract, XmlWriterDelegator xmlWriter, Object obj, RuntimeTypeHandle originalDeclaredTypeHandle, Type graphType)
   at System.Runtime.Serialization.DataContractSerializer.InternalWriteObjectContent(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver)
   at System.Runtime.Serialization.DataContractSerializer.InternalWriteObject(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver)
   at System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator writer, Object graph, DataContractResolver dataContractResolver)
   at Microsoft.ServiceFabric.Services.Remoting.V2.PooledBufferMessageBodySerializer`2.Microsoft.ServiceFabric.Services.Remoting.V2.IServiceRemotingResponseMessageBodySerializer.Serialize(IServiceRemotingResponseMessageBody serviceRemotingResponseMessageBody)
   at Microsoft.ServiceFabric.Services.Remoting.V2.FabricTransport.Runtime.FabricTransportMessageHandler.CreateFabricTransportMessage(IServiceRemotingResponseMessage retval, Int32 interfaceId, Stopwatch stopwatch)
   at Microsoft.ServiceFabric.Services.Remoting.V2.FabricTransport.Runtime.FabricTransportMessageHandler.<RequestResponseAsync>d__7.MoveNext()

When debugging it, I see that the call gets into the method implementation on a service. So it looks like it's proxy failing to deserialize the response despite both service and proxy are created with UseWrappedMessage = true as per instructions.

Is there something I am missing to make it work or is this a bug? I am interested in using interface compatibility and ability to decouple service interfaces from IService that should be supported with V2_1 but so far can't make it work even with this minimal example.

Thanks!

[vskh]

I've debugged this issue further and it seems that it's a serializer on the service side throwing exception i.e. after invoking the listener method and before sending out response. It appears that DataContractSerializer expects object type to match the underlying type of generated contract to match but they don't. As DataContract says body should be WrappedRemotingMessageBody but it seems to still get ServiceRemotingResponseMessageBody (despite setting UseWrappedMessage set to true).

I found no relevant unit tests in https://github.com/microsoft/service-fabric-services-and-actors-dotnet. Is there a way to make it work or it's a bug and it never worked at all?

[vskh]

Ok, I think I found the issue - it is because ServiceRemotingMessageDispatcher ignores UseWrappedMessage setting and just always uses DataContractRemotingMessageFactory as default which always produces ServiceRemotingResponseMessageBody.

[vskh]

I've updated repo with a workaround.

[masnider]

Thanks @vskh

@BharatNarasimman and @amanbha how do you want to follow up with this?

[vskh]

Thanks for looking into this. I checked out what it would take to fix it and seems like a bit of refactoring is required to make dispatcher properly use the message factory and it might be a change in API of existing classes so I decided to wait for someone from the team to look.

[xMarkos]

I ran into this problem, and the solution presented by @vskh solved it, however, I am confused why this bug still exists now, 3 years later after it was reported. Perhaps I am missing something fundamental here.