Exception thrown "The converter 'Schema.NET.ValuesJsonConverter' read too much or not enough."

[dermotblairca]

Describe the bug

When deserializing certain json from websites, SchemaSerializer.DeserializeObject throws the following exception "The converter 'Schema.NET.ValuesJsonConverter' read too much or not enough.". This is being thrown from System.Text.Json with the following stack trace

at System.Text.Json.ThrowHelper.ThrowJsonException_SerializationConverterRead(JsonConverter converter)
   at System.Text.Json.Serialization.JsonConverter`1.VerifyRead(JsonTokenType tokenType, Int32 depth, Int64 bytesConsumed, Boolean isValueConverter, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at Schema.NET.SchemaSerializer.DeserializeObject[T](String value) in C:\Github\Schema.NET\Source\Common\SchemaSerializer.cs:line 52
   at Schema.NET.Test.Examples.ProductTest.Deserializing_ProductJsonLd_HasComment() in C:\Github\Schema.NET\Tests\Schema.NET.Test\Examples\ProductTest.cs:line 91

Steps to reproduce

Run the following unit test:

[Fact]
public void Deserializing_ProductJsonLd_HasComment()
{
    var productJson = "{\"@context\":\"http://schema.org\",\"@type\":\"Product\",\"review\":[[]]}";

    var exceptionMessage = string.Empty;
    try
    {
        var product = SchemaSerializer.DeserializeObject<Product>(productJson);
    }
    catch (System.Text.Json.JsonException ex)
    {
        exceptionMessage = ex.Message;
    }

    Assert.Equal(string.Empty, exceptionMessage);
}

I got the above json from the following webpage: https://www.nahdionline.com/ar/koleston-hair-color-natural-black-developer-302-0 Schema.org validator does not show any issues with it: https://validator.schema.org/#url=https%3A%2F%2Fwww.nahdionline.com%2Far%2Fkoleston-hair-color-natural-black-developer-302-0

Expected behaviour

SchemaSerializer.DeserializeObject returns an object and does not throw an exception.

Schema objects

https://schema.org/Product (but affecting all objects)

[Turnerj]

Not sure what is going on there - maybe an issue with the encoding/unicode but at the same time, I kinda doubt it as I'd imagine the core runtime is pretty rock solid around Unicode handling by now.

One part that does strike me as odd is this: \"review\":[[]] I'd be surprised though if that was causing the issue.

[RehanSaeed]

@dermotblairca Does it work if you remove that piece of the JSON? Can you slowly remove bits of the JSON to narrow down the part causing the issue?

[dermotblairca]

Thanks for your suggestions both! Yes it is the "review" property, it works fine without this. I have updated the issue with a smaller piece of json that reproduces it.

[RehanSaeed]

Thanks for helping to narrow things down. The stack trace suggests that this is a problem deep in System.Text.Json, so it may be worth searching for an issue there or maybe raising a new one and linking here, so we can track it.

[dermotblairca]

Yes I will search for an issue there or else I will create one and link it back here. Thanks.

[dermotblairca]

I couldn't find a similar issue on dotnet repo so I tried to reproduce the issue just using System.Text.Json. However it seems to work ok with the simple example below. It deserializes the json fine:

using System.Text.Json;
using System.Text.Json.Serialization;

namespace SchemaNetBugReproduce
{
    class Program
    {
        static void Main(string[] args)
        {
            var DefaultSerializationSettings = new JsonSerializerOptions
            {
                AllowTrailingCommas = true,
                DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingDefault,
                Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
            };

            var productJson = "{\"Context\":\"http://schema.org\",\"Type\":\"Product\",\"Review\":[[]]}";

            var exceptionMessage = string.Empty;
            Product product = null;
            try
            {
                product = JsonSerializer.Deserialize<Product>(productJson, DefaultSerializationSettings);
            }
            catch (Exception ex)
            {
                exceptionMessage = ex.Message;
            }

            Console.WriteLine($"Exceptin: {exceptionMessage}");
        }
    }
}

public class Thing
{
    public string Context { get; set; }
    public string Type { get; set; }
    public List<List<string>> Review { get; set; }
}

public class Product : Thing
{  
}

I am using .Net Core 6.0 and System.Text.Json 6.0