Elsa.Exceptions.CannotSetActivityPropertyValueException: An exception was thrown whilst setting 'ForEach.Items'.Failed to evaluate expression

[jayachandra21]

Hello @sfmskywalker,

I have a custom activity whose return type as "DataRowCollection".

[ActivityOutput(DefaultWorkflowStorageProvider = TransientWorkflowStorageProvider.ProviderName)]
        public DataRowCollection Output { get; set; }

So I have designed a workflow such that forEach loop activity will read the data from custom activity.

So in this case getting the following error at ForEach loop activity:

[WRN] Failed to run workflow 39fe328944604e83aebfb37fcc298af0 Elsa.Exceptions.CannotSetActivityPropertyValueException: An exception was thrown whilst setting 'ForEach.Items'. See the inner exception for further details. ---> Elsa.Exceptions.ExpressionEvaluationException: Failed to evaluate expression ---> System.InvalidCastException: Object must implement IConvertible. at System.Convert.ChangeType(Object value, Type conversionType, IFormatProvider provider)

For this scenario using the Elsa Version 2.6.0 But same thing is working in Elsa Version 2.4

[jayachandra21]

Even I have tried of reading the output of custom activity into a workflow variable. But in that case as well getting the error as below

2022-04-18 17:41:14.777 +03:00 [ERR] Connection ID "14195346061442188274", Request ID "8000b124-0008-c500-b63f-84710c7967bb": An unhandled exception was thrown by the application.
Newtonsoft.Json.JsonSerializationException: Cannot create and populate list type System.Data.DataRowCollection. Path 'variables.data.ValOutput.$values', line 13, position 20.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewList(JsonReader reader, JsonArrayContract contract, Boolean& createdFromNonDefaultCreator)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ReadMetadataProperties(JsonReader reader, Type& objectType, JsonContract& contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue, Object& newValue, String& id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateDictionary(IDictionary dictionary, JsonReader reader, JsonDictionaryContract contract, JsonProperty containerProperty, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at Elsa.Persistence.EntityFramework.Core.Stores.EntityFrameworkWorkflowInstanceStore.OnLoading(ElsaContext dbContext, WorkflowInstance entity)
   at Elsa.Persistence.EntityFramework.Core.Stores.EntityFrameworkStore`2.ReadShadowProperties(TContext dbContext, T entity)
   at Elsa.Persistence.EntityFramework.Core.Stores.EntityFrameworkStore`2.<>c__DisplayClass12_1.<FindManyAsync>b__1(T x)
   at System.Linq.Enumerable.SelectListIterator`2.ToList()
   at Elsa.Persistence.EntityFramework.Core.Stores.EntityFrameworkStore`2.<>c__DisplayClass12_0.<<FindManyAsync>b__0>d.MoveNext()

[jayachandra21]

Taking reference of the below bug, For the Elsa 2.6.0 version...custom activity output return should it be a IConvertable before using it in the Foreach loop and Variable activity?

[(https://github.com/elsa-workflows/elsa-core/issues/2754)]

[mohdali]

@sfmskywalker can you please advise on this? DataSet and DataTable implement IListStore, can that be utilized in ForEach instead?

We have our custom activity to connect to Oracle database and we are also trying with the new SQL Activity. It seems something has changed after 2.4 that causes the issue.

[sfmskywalker]

Hi @jayachandra21 @mohdali

The serialization thing is quite tricky for me to get right without seeing a full repro, but if you can send me a sample project with this issue then I might be able to troubleshoot.

[jayachandra21]

Hi @sfmskywalker,

I have attached the project source code for refefence. In this project source code I have tried with Custom Activity output type as DataRowCollection and "DataTable" ElsaQuickstarts.Server.DashboardAndServer1 (4).zip

[jayachandra21]

Hello @sfmskywalker, I have shared the sample code which I have tried for using the ForEach loop. Please check the same and suggest me the right approach for the same.

I am referring the Elsa 2.6.0 version.

[sfmskywalker]

Hi @jayachandra21 - Thank you for the great repro - I am looking into this issue.

@constantine-v As it turns out, the same issue applies to the new ExecuteSqlQuery activity. I'll see if we can add support for IListSource.

[sfmskywalker]

Although I can successfully convert DataSet and DataTable to a collection, the next problem is that of serialization - the list item type of DataSet/DataTable.GetList() is System.Data.DataViewManagerListItemTypeDescriptor - which does not seem to support de-serialization due to the fact that there's seemingly no public constructor.

A workaround for this (if you want to use DataSet as an input to another activity) is to configure the output of the SQL activities to be Transient. The same must be done for the ForEach activity, where both input & output properties must be Transient.

An alternative approach would be to write our own set of DTO models that model a DataSet, DataTable, DataRow and DataColumn, one that is serializable.

Or perhaps there's a JSON converter available that supports DataSet serialization.

[sfmskywalker]

I just found this blog post from Rick Strahl describing the process of serializing DataSet/DataTable/DataRow objects using custom converters.

However, it only deals with writing JSON, not reading it back into actual DataSet/DataTable.DataRow objects (which is complicated).

[sfmskywalker]

In summary:

• I pushed an update that includes a new value converter that converts IListSource to ICollection<object>, allowing for DataSet and DataTable to be returned from Jint into the target activity property (Output in the case of the custom activity described here and the ExecuteSqlQuery activity).
• To use DataSet and DataTable types in a worflow, one must ensure that all input and output properties dealing with these values are marked as Transient. Without this, the workflow instance will no longer be deserializable.
• We might try and implement JSON converters for these types or consider implementing a simplified model for DataSet/DataTable/DataRow.

For Elsa 3, we will probably not have this issue, since input/output will always be passed transiently. It will be up to the workflow user to capture values into variables. The user must then make sure that what they store is serializable. The biggest difference with Elsa 2 being that in Elsa 2, we try to deal with values and serialization implicitly (which is tricky to do), while in Elsa 3, we let the user deal with serialization themselves explicitly. Although this does place the burden with the user, at the same time it makes the process a lot clearer and allows for greater control and flexibility I think.

[mohdali]

@sfmskywalker we faced a similar issue with a custom activity that had a property that wasn't serializable and we resolved the same way by forcing it to be transient.

Is there a way to prevent the workflow server from crashing due to the serialization issue? It feels very risky that one misconfigured activity can take the whole thing down.

Please let me know your thoughts.

[jayachandra21]

Hi @sfmskywalker, I have faced same type of issue as @mohdali san is mentioned. Workflow server got crashed and not be able to open the dashboard due to serialization issue.

Please advise.

[sfmskywalker]

@mohdali @jayachandra21

You're right, the server shouldn't crash when deserialization of a workflow instance fails. I have pushed an update that should fix this.

[mohdali]

@sfmskywalker Thanks a lot! Will check it out.