Failed to deserialize the Data property of class WorkflowInstance with MongoDb storage provider

[smadurange]

trafficstars

I'm using workflow in a AspNetCore web api. I setup services like

services.AddWorkflow(options =>
{
    options.UseMongoDB(mongoConnectionString, mongoDatabase);
});

I created a HelloWorld step like

public class HelloWorld : StepBody
{
    public override ExecutionResult Run(IStepExecutionContext context)
    {
        Console.WriteLine("Hello world");
        return ExecutionResult.Next();
    }
}

Then I create a definition in MongoDb which looks like:

{
    "_id": "5f86bd25471d8a36ace25e81",
    "ExternalId": "hello_world",
    "Version": {
        "$numberInt": "0"
    },
    "CustomProperty1": "Test1",
    "CustomProperty2": "Test2",
    "CustomProperty3": "Test3",
    "Definition": {
        "Id": "hello_world",
        "Version": {
            "$numberInt": "0"
        },
        "Description": null,
        "Steps": [
            {
                "StepType": "HelloWorld",
                "Id": "HelloWorld",
                "Name": "Say hello",
                "Do": [],
                "NextStepId": null,
                "Inputs": {},
                "Outputs": {}
            }
        ]
    }
}

When I try to start the workflow using the controller action:

[HttpPost("{id}")]
public async Task<IActionResult> Post(string id, [FromBody] ExpandoObject data)
{
    var instanceId = await _workflowController.StartWorkflow(id, data);
    return Ok();
}

I get the error:

Error executing item 5f86bd50471d8a36ace25e82 - An error occurred while deserializing the Data property of class WorkflowCore.Models.WorkflowInstance: _v element missing. An error occurred while deserializing the Data property of class WorkflowCore.Models.WorkflowInstance: _v element missing.

Exception:

2020-10-14 16:56:48.7390|1.0.0.0||WorkflowConsumer|System.FormatException: An error occurred while deserializing the Data property of class WorkflowCore.Models.WorkflowInstance: _v element missing.
 ---> System.FormatException: _v element missing.
   at MongoDB.Bson.Serialization.Serializers.ObjectSerializer.DeserializeDiscriminatedValue(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.ObjectSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonSerializer.Deserialize(IBsonReader bsonReader, Type nominalType, Action`1 configurator)
   at WorkflowCore.Persistence.MongoDB.Services.DataObjectSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.DeserializeMemberValue(BsonDeserializationContext context, BsonMemberMap memberMap)
   --- End of inner exception stack trace ---
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.DeserializeMemberValue(BsonDeserializationContext context, BsonMemberMap memberMap)
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.DeserializeClass(BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize[TValue](IBsonSerializer`1 serializer, BsonDeserializationContext context)
   at MongoDB.Driver.Core.Operations.CursorBatchDeserializationHelper.DeserializeBatch[TDocument](RawBsonArray batch, IBsonSerializer`1 documentSerializer, MessageEncoderSettings messageEncoderSettings)
   at MongoDB.Driver.Core.Operations.FindCommandOperation`1.CreateFirstCursorBatch(BsonDocument cursorDocument)
   at MongoDB.Driver.Core.Operations.FindCommandOperation`1.CreateCursor(IChannelSourceHandle channelSource, BsonDocument commandResult)
   at MongoDB.Driver.Core.Operations.FindCommandOperation`1.ExecuteAsync(RetryableReadContext context, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(RetryableReadContext context, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(IReadBinding binding, CancellationToken cancellationToken)
   at MongoDB.Driver.OperationExecutor.ExecuteReadOperationAsync[TResult](IReadBinding binding, IReadOperation`1 operation, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoCollectionImpl`1.ExecuteReadOperationAsync[TResult](IClientSessionHandle session, IReadOperation`1 operation, ReadPreference readPreference, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)
   at WorkflowCore.Persistence.MongoDB.Services.MongoPersistenceProvider.GetWorkflowInstance(String Id)
   at WorkflowCore.Services.BackgroundTasks.WorkflowConsumer.ProcessItem(String itemId, CancellationToken cancellationToken)
   at WorkflowCore.Services.BackgroundTasks.WorkflowConsumer.ProcessItem(String itemId, CancellationToken cancellationToken)
   at WorkflowCore.Services.BackgroundTasks.QueueConsumer.ExecuteItem(String itemId, EventWaitHandle waitHandle)
System.FormatException: _v element missing.
   at MongoDB.Bson.Serialization.Serializers.ObjectSerializer.DeserializeDiscriminatedValue(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.ObjectSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonSerializer.Deserialize(IBsonReader bsonReader, Type nominalType, Action`1 configurator)
   at WorkflowCore.Persistence.MongoDB.Services.DataObjectSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.DeserializeMemberValue(BsonDeserializationContext context, BsonMemberMap memberMap)
2020-10-14 16:56:57.4648|1.0.0.0||WorkflowConsumer|System.FormatException: An error occurred while deserializing the Data property of class WorkflowCore.Models.WorkflowInstance: _v element missing.
 ---> System.FormatException: _v element missing.
   at MongoDB.Bson.Serialization.Serializers.ObjectSerializer.DeserializeDiscriminatedValue(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.ObjectSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonSerializer.Deserialize(IBsonReader bsonReader, Type nominalType, Action`1 configurator)
   at WorkflowCore.Persistence.MongoDB.Services.DataObjectSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.DeserializeMemberValue(BsonDeserializationContext context, BsonMemberMap memberMap)
   --- End of inner exception stack trace ---
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.DeserializeMemberValue(BsonDeserializationContext context, BsonMemberMap memberMap)
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.DeserializeClass(BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize[TValue](IBsonSerializer`1 serializer, BsonDeserializationContext context)
   at MongoDB.Driver.Core.Operations.CursorBatchDeserializationHelper.DeserializeBatch[TDocument](RawBsonArray batch, IBsonSerializer`1 documentSerializer, MessageEncoderSettings messageEncoderSettings)
   at MongoDB.Driver.Core.Operations.FindCommandOperation`1.CreateFirstCursorBatch(BsonDocument cursorDocument)
   at MongoDB.Driver.Core.Operations.FindCommandOperation`1.CreateCursor(IChannelSourceHandle channelSource, BsonDocument commandResult)
   at MongoDB.Driver.Core.Operations.FindCommandOperation`1.ExecuteAsync(RetryableReadContext context, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(RetryableReadContext context, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.FindOperation`1.ExecuteAsync(IReadBinding binding, CancellationToken cancellationToken)
   at MongoDB.Driver.OperationExecutor.ExecuteReadOperationAsync[TResult](IReadBinding binding, IReadOperation`1 operation, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoCollectionImpl`1.ExecuteReadOperationAsync[TResult](IClientSessionHandle session, IReadOperation`1 operation, ReadPreference readPreference, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)
   at WorkflowCore.Persistence.MongoDB.Services.MongoPersistenceProvider.GetWorkflowInstance(String Id)
   at WorkflowCore.Services.BackgroundTasks.WorkflowConsumer.ProcessItem(String itemId, CancellationToken cancellationToken)
   at WorkflowCore.Services.BackgroundTasks.WorkflowConsumer.ProcessItem(String itemId, CancellationToken cancellationToken)
   at WorkflowCore.Services.BackgroundTasks.QueueConsumer.ExecuteItem(String itemId, EventWaitHandle waitHandle)
System.FormatException: _v element missing.
   at MongoDB.Bson.Serialization.Serializers.ObjectSerializer.DeserializeDiscriminatedValue(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.ObjectSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonSerializer.Deserialize(IBsonReader bsonReader, Type nominalType, Action`1 configurator)
   at WorkflowCore.Persistence.MongoDB.Services.DataObjectSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
   at MongoDB.Bson.Serialization.BsonClassMapSerializer`1.DeserializeMemberValue(BsonDeserializationContext context, BsonMemberMap memberMap)

[danielgerlag]

It looks like your definition JSON does not match the schema required, (ie. DataType not set) see https://workflow-core.readthedocs.io/en/latest/json-yaml/ For MongoDB, you might also need to Register BSON class maps for any custom classes you are storing inside the workflow.

[smadurange]

Hmm, I set the DataType like so

public WorkflowDefinition Convert(Definition source)
{
    if (source == null) throw new ArgumentNullException(nameof(source));

    var dataType = typeof(ExpandoObject);

    var result = new WorkflowDefinition
    {
        Id = source.Id,
        Version = source.Version,
        Description = source.Description,
        Steps = ConvertSteps(source.Steps),
        DefaultErrorBehavior = DefaultErrorBehavior,
        DefaultErrorRetryInterval = _defaultErrorRetryInterval,
        DataType = dataType
    };

    return result;
}

So, I'd just like to save the ExpandObject as it is to Mongo. As a workaround, I copied the WorkflowPersistenceProvider from your Conductor project (great work, btw. Really enjoy working with your apis!) and implemented my DataObjectSerializer like so

public class DataObjectSerializer : SerializerBase<object>
{
    public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, object value) =>
        BsonSerializer.Serialize(context.Writer, BsonHelper.ConvertToBsonDocument(value));

    public override object Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args) =>
        BsonSerializer.Deserialize<ExpandoObject>(context.Reader);
}

It'd just be nicer if I could use the MonoDB provider from this project instead of rolling my own like above. So, I'm wondering if I'm doing something wrong or the persistence provider just doesn't let me use ExpandObject as the type of my data.

[VarsikH]

Hello I have same problem I alerady add mongo client and when after that I use services.Add Workflow(x => x.User MongoDB(@"mongodb://localhost:27017", "workflow")); It is not work. How I can configure that if I already have added mongo client. When I already have added Mongo client Workflowcore can not get Flow data in database