quartznet
STJ serializer does not support objects in `JobDataMap`
Describe the bug
The new System.Text.Json serializer does not support non-primitive non-IDictionary values inside JobDataMap. This is a regression compares to the Newtonsoft serializer, which used TypeNameHandling.Auto, and there is no easy way to override the new behavior (it is set by new custom converters which are internal).
Version used
3.11.0
To Reproduce
Create a job with objects inside the job data, for example:
public static IJobDetail CreateJob(string id, ICollection<MyDto> data)
{
var jobData = new JobDataMap
{
["JobType"] = JobType.Inline,
["MyData"] = data
};
return JobBuilder.Create<MyJob>()
.WithIdentity(id)
.SetJobData(jobData)
.RequestRecovery()
.StoreDurably(false)
.Build();
}
Schedule the job with any trigger. When the trigger fires, the deserialization will fail with an exception similar to this:
Quartz.JsonSerializationException: Could not deserialize JSON: {"JobType":0,"MyData":[{...},{...}]}
---> Quartz.JsonSerializationException: Failed to parse JobDataMap from json
---> System.Text.Json.JsonException: Unsupported value kind: Array
at Quartz.Util.Utf8JsonWriterExtensions.GetJobDataMap(JsonElement jsonElement, JsonSerializerOptions options)
at Quartz.Converters.DictionaryConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
--- End of inner exception stack trace ---
at Quartz.Converters.DictionaryConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
at System.Text.Json.JsonSerializer.Deserialize[TValue](ReadOnlySpan`1 utf8Json, JsonSerializerOptions options)
at Quartz.Simpl.SystemTextJsonObjectSerializer.DeSerialize[T](Byte[] data)
[See nested exception: Quartz.JsonSerializationException: Failed to parse JobDataMap from json
---> System.Text.Json.JsonException: Unsupported value kind: Array
at Quartz.Util.Utf8JsonWriterExtensions.GetJobDataMap(JsonElement jsonElement, JsonSerializerOptions options)
at Quartz.Converters.DictionaryConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
--- End of inner exception stack trace ---
at Quartz.Converters.DictionaryConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
at System.Text.Json.JsonSerializer.Deserialize[TValue](ReadOnlySpan`1 utf8Json, JsonSerializerOptions options)
at Quartz.Simpl.SystemTextJsonObjectSerializer.DeSerialize[T](Byte[] data) [See nested exception: System.Text.Json.JsonException: Unsupported value kind: Array
at Quartz.Util.Utf8JsonWriterExtensions.GetJobDataMap(JsonElement jsonElement, JsonSerializerOptions options)
at Quartz.Converters.DictionaryConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)]]
--- End of inner exception stack trace ---
at Quartz.Impl.AdoJobStore.JobStoreSupport.RetrieveJob(ConnectionAndTransactionHolder conn, JobKey jobKey, CancellationToken cancellationToken)
at Quartz.Impl.AdoJobStore.JobStoreSupport.TriggerFired(ConnectionAndTransactionHolder conn, IOperableTrigger trigger, CancellationToken cancellationToken)
...
Expected behavior
Either the behavior should be similar to Newtonsoft, or (ideally) there should be a way to opt-in to serialization of type information.
Additional context
The exception thrown when attempting to deserialize varies depending on the exact settings and objects used. Using ReferenceHandler.Preserve causes additional issues due to the custom parsing encountering the $value field. Overriding Utf8JsonWriterExtensions.GetJobDataMap is not enough to fix this. Instead, removing about half of the custom converters entirely may be the best way to switch the behavior to match the old serializer.
The Newtonsoft serializer would serialize most things as-is without any custom converters, so I don't really understand what the purpose of the new custom converters is. I don't feel comfortable blindly removing them, although it would be the best solution for my use case.
Is it related to compatibility with older versions of STJ? Compatibility with JSON from previous versions? Some other reason?
The serialization logic was recently improved to support 1:1 results *when using the recommend useProperties = true setup between Newtonsoft and STJ by @VilleHakli in #2444 . So the the STJ behavior which is is heavily tested is the safe string-based values currently.
If you want the more unsafe behavior without converters then I guess the way to go is to customize the CreateOptions like documented.
I'm closing this as I don't want to advocate using objects persisted in job data map when needing serialization. If you really need this, be prepared to customize serialization. Simple keys and values will give you more leg room when migrating data and models.