runtime
runtime copied to clipboard
Published
20 hours ago •
dotnet
dotnet
System.Text.Json failing some large file tests
https://dev.azure.com/dnceng/public/_build/results?buildId=1387876&view=ms.vss-test-web.build-test-results-tab&runId=40401306&resultId=194463&paneView=debug
If I didn't know any better, I'd say "memory corruption".
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-59626-merge-ff748885001649f2ad/System.Text.Json.Tests/1/console.55b96e63.log?sv=2019-07-07&se=2021-10-17T15%3A49%3A24Z&sr=c&sp=rl&sig=jkQ9vlfitTdm9LAJkyCb4ZnZxrI5Gb%2F8qvt%2FSn0S27M%3D
Console log: 'System.Text.Json.Tests' from job ff748885-0016-49f2-ad26-20905cd95f3d workitem 0045d34b-f5a3-424a-8789-b58e57023e81 (osx.1015.amd64.open) executed on machine dci-mac-build-195
+ ./RunTests.sh --runtime-path /tmp/helix/working/A15608F4/p
----- start Mon Sep 27 11:49:52 EDT 2021 =============== To repro directly: =====================================================
pushd .
/tmp/helix/working/A15608F4/p/dotnet exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
popd
===========================================================================================================
/private/tmp/helix/working/A15608F4/w/A1BD08B8/e /private/tmp/helix/working/A15608F4/w/A1BD08B8/e
Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Text.Json.Tests (found 2776 of 2815 test cases)
Starting: System.Text.Json.Tests (parallel test collections = on, max threads = 12)
System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes(formatted: False, skipValidation: False) [FAIL]
Assert.Equal() Failure
↓ (pos 1102164)
Expected: ···qKioqKioqKioqKioqKiosKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq···
Actual: ···qKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq···
↑ (pos 1102164)
Stack Trace:
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonTestHelper.cs(738,0): at System.Text.Json.JsonTestHelper.AssertContentsAgainstJsonNet(String expectedValue, String value, Boolean skipSpecialRules)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonTestHelper.cs(714,0): at System.Text.Json.JsonTestHelper.AssertContents(String expectedValue, ArrayBufferWriter`1 buffer, Boolean skipSpecialRules)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Utf8JsonWriterTests.cs(3202,0): at System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes(Boolean formatted, Boolean skipValidation)
System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers(formatted: False, skipValidation: False, keyString: ">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>") [FAIL]
Assert.Equal() Failure
↓ (pos 484474)
Expected: ···e\u003e\u003e\u003e\w003e\u003e\u003e\u003e\u003e\u003e\u003e···
Actual: ···e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e···
↑ (pos 484474)
Stack Trace:
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonTestHelper.cs(738,0): at System.Text.Json.JsonTestHelper.AssertContentsAgainstJsonNet(String expectedValue, String value, Boolean skipSpecialRules)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonTestHelper.cs(714,0): at System.Text.Json.JsonTestHelper.AssertContents(String expectedValue, ArrayBufferWriter`1 buffer, Boolean skipSpecialRules)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Utf8JsonWriterTests.cs(5617,0): at System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers(Boolean formatted, Boolean skipValidation, String keyString)
System.Text.Json.Serialization.Tests.StreamTests_Sync.HandleCollectionsAsync [FAIL]
Assert.Equal() Failure
↓ (pos 12025)
Expected: ···eeeeeeeeeeeeeeeeeeeegeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee···
Actual: ···eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee···
↑ (pos 12025)
Stack Trace:
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(61,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(45,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(45,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(20,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(102,0): at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TElement](Stream memoryStream, String expectedJson, Type type, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(71,0): at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TElement](Object obj, Type type, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(53,0): at System.Text.Json.Serialization.Tests.StreamTests.RunTestAsync[TElement]()
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(25,0): at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync()
--- End of stack trace from previous location ---
Finished: System.Text.Json.Tests
=== TEST EXECUTION SUMMARY ===
System.Text.Json.Tests Total: 20247, Errors: 0, Failed: 3, Skipped: 0, Time: 58.695s
- System.Text.Json.Serialization.Tests.StreamTests_Sync.HandleCollectionsAsync
- System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes(formatted: False, skipValidation: False)
- System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers(formatted: False, skipValidation: False, keyString: ">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
Error Message
Fill the error message using known issues guidance.
{
"ErrorMessage": "System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers",
"BuildRetry": false
}
Report
Summary
| 24-Hour Hit Count | 7-Day Hit Count | 1-Month Count |
|---|---|---|
| 0 | 7 | 23 |
Known issue validation
Build: :mag_right: Result validation: :warning: Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: :mag_right:" line.
Tagging subscribers to this area: @eiriktsarpalis, @layomia See info in area-owners.md if you want to be subscribed.
Issue Details
https://dev.azure.com/dnceng/public/_build/results?buildId=1387876&view=ms.vss-test-web.build-test-results-tab&runId=40401306&resultId=194463&paneView=debug
If I didn't know any better, I'd say "memory corruption".
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-59626-merge-ff748885001649f2ad/System.Text.Json.Tests/1/console.55b96e63.log?sv=2019-07-07&se=2021-10-17T15%3A49%3A24Z&sr=c&sp=rl&sig=jkQ9vlfitTdm9LAJkyCb4ZnZxrI5Gb%2F8qvt%2FSn0S27M%3D
Console log: 'System.Text.Json.Tests' from job ff748885-0016-49f2-ad26-20905cd95f3d workitem 0045d34b-f5a3-424a-8789-b58e57023e81 (osx.1015.amd64.open) executed on machine dci-mac-build-195
+ ./RunTests.sh --runtime-path /tmp/helix/working/A15608F4/p
----- start Mon Sep 27 11:49:52 EDT 2021 =============== To repro directly: =====================================================
pushd .
/tmp/helix/working/A15608F4/p/dotnet exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
popd
===========================================================================================================
/private/tmp/helix/working/A15608F4/w/A1BD08B8/e /private/tmp/helix/working/A15608F4/w/A1BD08B8/e
Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Text.Json.Tests (found 2776 of 2815 test cases)
Starting: System.Text.Json.Tests (parallel test collections = on, max threads = 12)
System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes(formatted: False, skipValidation: False) [FAIL]
Assert.Equal() Failure
↓ (pos 1102164)
Expected: ···qKioqKioqKioqKioqKiosKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq···
Actual: ···qKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq···
↑ (pos 1102164)
Stack Trace:
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonTestHelper.cs(738,0): at System.Text.Json.JsonTestHelper.AssertContentsAgainstJsonNet(String expectedValue, String value, Boolean skipSpecialRules)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonTestHelper.cs(714,0): at System.Text.Json.JsonTestHelper.AssertContents(String expectedValue, ArrayBufferWriter`1 buffer, Boolean skipSpecialRules)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Utf8JsonWriterTests.cs(3202,0): at System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes(Boolean formatted, Boolean skipValidation)
System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers(formatted: False, skipValidation: False, keyString: ">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>") [FAIL]
Assert.Equal() Failure
↓ (pos 484474)
Expected: ···e\u003e\u003e\u003e\w003e\u003e\u003e\u003e\u003e\u003e\u003e···
Actual: ···e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e···
↑ (pos 484474)
Stack Trace:
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonTestHelper.cs(738,0): at System.Text.Json.JsonTestHelper.AssertContentsAgainstJsonNet(String expectedValue, String value, Boolean skipSpecialRules)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/JsonTestHelper.cs(714,0): at System.Text.Json.JsonTestHelper.AssertContents(String expectedValue, ArrayBufferWriter`1 buffer, Boolean skipSpecialRules)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Utf8JsonWriterTests.cs(5617,0): at System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers(Boolean formatted, Boolean skipValidation, String keyString)
System.Text.Json.Serialization.Tests.StreamTests_Sync.HandleCollectionsAsync [FAIL]
Assert.Equal() Failure
↓ (pos 12025)
Expected: ···eeeeeeeeeeeeeeeeeeeegeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee···
Actual: ···eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee···
↑ (pos 12025)
Stack Trace:
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(61,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(45,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(45,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(20,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(102,0): at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TElement](Stream memoryStream, String expectedJson, Type type, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(71,0): at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TElement](Object obj, Type type, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(53,0): at System.Text.Json.Serialization.Tests.StreamTests.RunTestAsync[TElement]()
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(25,0): at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync()
--- End of stack trace from previous location ---
Finished: System.Text.Json.Tests
=== TEST EXECUTION SUMMARY ===
System.Text.Json.Tests Total: 20247, Errors: 0, Failed: 3, Skipped: 0, Time: 58.695s
- System.Text.Json.Serialization.Tests.StreamTests_Sync.HandleCollectionsAsync
- System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes(formatted: False, skipValidation: False)
- System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers(formatted: False, skipValidation: False, keyString: ">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
| Author: | bartonjs |
|---|---|
| Assignees: | - |
| Labels: |
|
| Milestone: | - |
Interesting observation: In all three failures the expected character vs the actual char had a difference of 2. (q => s, u => w, e => g).
Maybe it really is some sort of memory corruption (something has a bad pointer somewhere, and added 2).
Secondary observation: They all feel like they have expected and actual backwards :smile:.
The runtime-extra-platforms rolling build also hit the failure in Writing3MBBase64Bytes (on Windows Server 2022): https://dev.azure.com/dnceng/public/_build/results?buildId=1612202&view=ms.vss-test-web.build-test-results-tab&runId=44896660&resultId=189854&paneView=debug
Not sure if this is exactly related, but build https://dev.azure.com/dnceng/public/_build/results?buildId=1697303&view=logs&j=05e92ac1-194e-59cf-664a-fa72d1cdd19b&t=caea1d4b-c90c-5be1-e57d-c4635079c333 failed with HandleCollectionsAsync test failing in a very similar way. Here are the logs: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-50e485d6cc7b461f9a/System.Text.Json.Tests/1/console.6fad303f.log?sv=2019-07-07&se=2022-04-23T08%3A48%3A09Z&sr=c&sp=rl&sig=SNdOcc025qirLoyBCvPN7U%2BHl1uIgU94Zo64yUp0CLk%3D
In @bartonjs case, the second least significant bit is flipping 0->1 in every case. If it was a bad pointer I think we'd expect 1->0 also, but it's a sample of three.
In @joperezr case (again as @bartonjs points out, Expected and Actual are reversed - cc @eiriktsarpalis ) it is quite different, several bits flipped, and it's then repeated.
Assert.Equal() Failure
↓ (pos 13390)
Expected: ···xxxxxxxxxxxxxxxxxxxxooooooooooooooooooooooooooooooooooooooooo···
Actual: ···xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx···
↑ (pos 13390)
Stack Trace:
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(61,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(45,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(45,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(55,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(20,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(87,0): at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(50,0): at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(30,0): at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType)
--- End of stack trace from previous location ---
System.Text.Json.Serialization.Tests.StreamTests_Sync.HandleCollectionsAsync<Collection<StreamTests+ImmutableStructWithStrings>, StreamTests+ImmutableStructWithStrings>(collection: [System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, ...], bufferSize: 16384, elementType: Witness`1 { }) [FAIL]
Assert.Equal() Failure
↓ (pos 8986)
Expected: ···ppppppppppppppppppppwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww···
Actual: ···ppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp···
↑ (pos 8986)
Stack Trace:
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(61,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(45,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual)
It's important to record machine names in such cases. Last one above was dci-mac-build-365.local. Looks like we lost the other two? We have in the past send a box to the trash because it was regularly flipping 1 bit.
Might be related to this other issue? https://github.com/dotnet/runtime/issues/65021
In the same CI job that Joe shared, there's another log file with a slightly different callstack, in case it's helpful:
Pipeline: Libraries Test Run release coreclr OSX x64 Release Test method:
System.Text.Json.Serialization.Tests.StreamTests_Async.HandleCollectionsAsync
<ConcurrentStack<StreamTests+ImmutableStructWithStrings>, StreamTests+ImmutableStructWithStrings>
(collection: [System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings,
System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings,
System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings,
System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings,
System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, ...],
bufferSize: 16384, elementType: Witness`1 { })
[FAIL]
Callstack:
System.Text.Json.JsonException : '0x00' is invalid within a JSON string. The string should be correctly escaped. Path: $[14].MySecondString | LineNumber: 0 | BytePositionInLine: 439315.
---- System.Text.Json.JsonReaderException : '0x00' is invalid within a JSON string. The string should be correctly escaped. LineNumber: 0 | BytePositionInLine: 439315.
Stack Trace:
/_/src/libraries/System.Text.Json/src/System/Text/Json/ThrowHelper.Serialization.cs(265,0): at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs(79,0): at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.Helpers.cs(43,0): at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs(324,0): at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs(48,0): at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(75,0): at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(50,0): at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(30,0): at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType)
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
/_/src/libraries/System.Text.Json/src/System/Text/Json/ThrowHelper.cs(279,0): at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1365,0): at System.Text.Json.Utf8JsonReader.ConsumeStringAndValidate(ReadOnlySpan`1 data, Int32 idx)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1271,0): at System.Text.Json.Utf8JsonReader.ConsumeString()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1032,0): at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(800,0): at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(278,0): at System.Text.Json.Utf8JsonReader.Read()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Object/ObjectWithParameterizedConstructorConverter.cs(40,0): at System.Text.Json.Serialization.Converters.ObjectWithParameterizedConstructorConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs(221,0): at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/JsonCollectionConverter.cs(116,0): at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs(246,0): at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs(62,0): at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
@dotnet/area-system-text-json Happened once again in today's morning rolling build: https://dev.azure.com/dnceng/public/_build/results?buildId=1700046&view=results
- Pipeline:
Libraries Test Run checked coreclr OSX x64 Release - Job: https://dev.azure.com/dnceng/public/_build/results?buildId=1700046&view=logs&j=5daca27f-52cd-532c-7da3-66cd43b2fb44&t=544b6965-1404-5848-fc5a-30d8345c68ff&l=57
- Log: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-dbec34c964934fa7a9/System.Text.Json.Tests/1/console.5cf88315.log?sv=2019-07-07&se=2022-04-25T08%3A46%3A32Z&sr=c&sp=rl&sig=m70Hub%2B8XzyMvABaFJij3y3kzs5ziKhzveyOKXgYg2A%3D
- Test method:
System.Text.Json.Serialization.Tests.StreamTests_Sync.HandleCollectionsAsync<ImmutableStack<StreamTests+ImmutableStructWithStrings>, StreamTests+ImmutableStructWithStrings>(collection: [System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, System.Text.Json.Serialization.Tests.StreamTests+ImmutableStructWithStrings, ...], bufferSize: 16384, elementType: Witness1 { }) [FAIL]` - Callstack:
System.Text.Json.JsonReaderException : '0x00' is invalid within a JSON string. The string should be correctly escaped. LineNumber: 0 | BytePositionInLine: 392105.
Stack Trace:
/_/src/libraries/System.Text.Json/src/System/Text/Json/ThrowHelper.cs(279,0): at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1365,0): at System.Text.Json.Utf8JsonReader.ConsumeStringAndValidate(ReadOnlySpan`1 data, Int32 idx)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1293,0): at System.Text.Json.Utf8JsonReader.ConsumeString()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1032,0): at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(800,0): at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(278,0): at System.Text.Json.Utf8JsonReader.Read()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Document/JsonDocument.cs(1077,0): at System.Text.Json.JsonDocument.Parse(ReadOnlySpan`1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Document/JsonDocument.Parse.cs(689,0): at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Document/JsonDocument.Parse.cs(261,0): at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 json, JsonDocumentOptions options)
/_/src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs(19,0): at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(87,0): at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(54,0): at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(33,0): at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType)
Logging machine name from above instance:
Console log: 'System.Text.Json.Tests' from job dbec34c9-6493-4fa7-a9d3-a219dadba558 workitem 351aad77-de0d-4213-bf7f-90d96b9c35c1 (osx.1200.amd64.open) executed on machine dci-mac-build-387.local
so no evidence this is hardware specific.
@eiriktsarpalis @steveharter @layomia can we get the test disabled or a fix submitted? This happened again in the runtime-extra-platforms rolling build from this morning and is blocking a clean CI.
- Rolling build: https://dev.azure.com/dnceng/public/_build/results?buildId=1701191&view=results
- Pipeline:
Libraries Test Run release coreclr OSX x64 Release - Job: https://dev.azure.com/dnceng/public/public%20Team/_build/results?buildId=1701191&view=logs&j=05e92ac1-194e-59cf-664a-fa72d1cdd19b&t=caea1d4b-c90c-5be1-e57d-c4635079c333&l=57
- Log: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-bbb9258279de4d6584/System.Text.Json.Tests/1/console.8de9c6bc.log?sv=2019-07-07&se=2022-04-25T22%3A04%3A39Z&sr=c&sp=rl&sig=NqsVe8OO2YrApFHyTkeDyCi1MyDgsd7%2FGrE65Y63Oq4%3D
- Test method:
System.Text.Json.Serialization.Tests.StreamTests_Sync.HandleCollectionsAsyncIssue: Callstack:
System.Text.Json.Serialization.Tests.StreamTests_Sync.HandleCollectionsAsync<ImmutableStack<String>, String>(collection: ["vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv"..., "oooooooooooooooooooooooooooooooooooooooooooooooooo"..., "ssssssssssssssssssssssssssssssssssssssssssssssssss"..., "nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn"..., "wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww"..., ...], bufferSize: 16384, elementType: Witness`1 { }) [FAIL]
System.Text.Json.JsonException : '0x00' is invalid within a JSON string. The string should be correctly escaped. Path: $[14] | LineNumber: 0 | BytePositionInLine: 216537.
---- System.Text.Json.JsonReaderException : '0x00' is invalid within a JSON string. The string should be correctly escaped. LineNumber: 0 | BytePositionInLine: 216537.
Stack Trace:
/_/src/libraries/System.Text.Json/src/System/Text/Json/ThrowHelper.Serialization.cs(265,0): at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs(79,0): at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.Helpers.cs(43,0): at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs(324,0): at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs(48,0): at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(54,0): at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options)
/_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs(33,0): at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType)
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
/_/src/libraries/System.Text.Json/src/System/Text/Json/ThrowHelper.cs(279,0): at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1365,0): at System.Text.Json.Utf8JsonReader.ConsumeStringAndValidate(ReadOnlySpan`1 data, Int32 idx)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1293,0): at System.Text.Json.Utf8JsonReader.ConsumeString()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1036,0): at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1801,0): at System.Text.Json.Utf8JsonReader.ConsumeNextToken(Byte marker)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(1689,0): at System.Text.Json.Utf8JsonReader.ConsumeNextTokenOrRollback(Byte marker)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(884,0): at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs(272,0): at System.Text.Json.Utf8JsonReader.Read()
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/JsonCollectionConverter.cs(198,0): at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs(221,0): at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs(62,0): at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
FWIW this seems like a different failure compared to the original report. Is this impacting osx x64 builds only? Should we only disable the test for that platform?
I'll add couple of other tests I've seen locally, first 3 look like the "not found" value is actually in the string and in all 3 cases it's using current culture for comparison:
System.Text.Json.SourceGeneration.Tests.SerializationContextTests.SerializeObjectArray_WithCustomOptions [FAIL]
Assert.Contains() Failure
Not found: organizationName
In value: [{"activeOrUpcomingEvents":[{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"},{"id":10,"imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","name":"Just a name","campaignName":"The very new campaign","campaignManagedOrganizerName":"Name FamilyName","description":"The .NET Foundation works with Microsoft and the broader industry to increase the exposure of open source projects in the .NET community and the .NET Foundation. The .NET Foundation provides access to these resources to projects and looks to promote the activities of our communities.","startDate":"2001-02-03T04:05:06.789+00:00","endDate":"2002-02-03T04:05:06.789+00:00","offset":"00:00:00"}],"featuredCampaign":{"id":234235,"title":"Promoting Open Source","description":"Very nice campaign","imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","organizationName":"The Company XYZ","headline":"The Headline"},"isNewAccount":false,"hasFeaturedCampaign":true},{"id":234235,"title":"Promoting Open Source","description":"Very nice campaign","imageUrl":"https://www.dotnetfoundation.org/theme/img/carousel/foundation-diagram-content.png","organizationName":"The Company XYZ","headline":"The Headline"}]
Stack Trace:
D:\src\runtime\src\libraries\System.Text.Json\tests\System.Text.Json.SourceGeneration.Tests\SerializationContextTests.cs(339,0): at System.Text.Json.SourceGeneration.Tests.SerializationContextTests.SerializeObjectArray_WithCustomOptions()
Finished: System.Text.Json.SourceGeneration.Roslyn4.0.Tests
System.Text.Json.Serialization.Tests.PolymorphicTests_Element.SimpleTestClassAsRootObject [FAIL]
Assert.Contains() Failure
Not found: "MyBooleanTrue":true
In value: {"MyInt16Array":[1],"MyInt32Array":[2],"MyInt64Array":[3],"MyUInt16Array":[4],"MyUInt32Array":[5],"MyUInt64Array":[6],"MyByteArray":"Bw==","MySByteArray":[8],"MyCharArray":["a"],"MyStringArray":["Hello"],"MyDecimalArray":[3.3],"MyBooleanTrueArray":[true],"MyBooleanFalseArray":[false],"MySingleArray":[1.1],"MyDoubleArray":[2.2],"MyDateTimeArray":["2019-01-30T12:01:02Z"],"MyEnumArray":[2],"MyStringList":["Hello"],"MyStringIEnumerable":["Hello"],"MyStringIList":["Hello"],"MyStringICollection":["Hello"],"MyStringIEnumerableT":["Hello"],"MyStringIListT":["Hello"],"MyStringICollectionT":["Hello"],"MyStringIReadOnlyCollectionT":["Hello"],"MyStringIReadOnlyListT":["Hello"],"MyStringISetT":["Hello"],"MyStringToStringKeyValuePair":{"Key":"myKey","Value":"myValue"},"MyStringToStringIDict":{"key":"value"},"MyStringToStringGenericDict":{"key":"value"},"MyStringToStringGenericIDict":{"key":"value"},"MyStringToStringGenericIReadOnlyDict":{"key":"value"},"MyStringToStringImmutableDict":{"key":"value"},"MyStringToStringIImmutableDict":{"key":"value"},"MyStringToStringImmutableSortedDict":{"key":"value"},"MyStringStackT":["World","Hello"],"MyStringQueueT":["Hello","World"],"MyStringHashSetT":["Hello"],"MyStringLinkedListT":["Hello"],"MyStringSortedSetT":["Hello"],"MyStringIImmutableListT":["Hello"],"MyStringIImmutableStackT":["Hello"],"MyStringIImmutableQueueT":["Hello"],"MyStringIImmutableSetT":["Hello"],"MyStringImmutableHashSetT":["Hello"],"MyStringImmutableArray":null,"MyStringImmutableListT":["Hello"],"MyStringImmutableStackT":["Hello"],"MyStringImmutablQueueT":["Hello"],"MyStringImmutableSortedSetT":["Hello"],"MyInt16":1,"MyInt32":2,"MyInt64":3,"MyUInt16":4,"MyUInt32":5,"MyUInt64":6,"MyByte":7,"MySByte":8,"MyChar":"a","MyString":"Hello","MyDecimal":3.3,"MyBooleanTrue":true,"MyBooleanFalse":false,"MySingle":1.1,"MyDouble":2.2,"MyDateTime":"2019-01-30T12:01:02Z","MyGuid":"5bb9d872-da8a-471e-aa70-08e19102683d","MyEnum":2,"MyStruct":{"One":1,"Two":3.14}}
Stack Trace:
D:\src\runtime\src\libraries\System.Text.Json\tests\System.Text.Json.Tests\Serialization\PolymorphicTests.cs(342,0): at System.Text.Json.Serialization.Tests.PolymorphicTests.SimpleTestClassAsRootObject()
System.Text.Json.Serialization.Tests.PolymorphicTests_Element.NestedObjectAsRootObject [FAIL]
Assert.Contains() Failure
Not found: "ImmutableQueueT":["Hello","World"]
In value: {"Address":{"City":"MyCity"},"List":["Hello","World"],"Array":["Hello","Again"],"NullableInt":42,"NullableIntArray":[null,42,null],"IEnumerable":["Hello","World"],"IList":["Hello","World"],"ICollection":["Hello","World"],"IEnumerableT":["Hello","World"],"IListT":["Hello","World"],"ICollectionT":["Hello","World"],"IReadOnlyCollectionT":["Hello","World"],"IReadOnlyListT":["Hello","World"],"ISetT":["Hello","World"],"StackT":["World","Hello"],"QueueT":["Hello","World"],"HashSetT":["Hello","World"],"LinkedListT":["Hello","World"],"SortedSetT":["Hello","World"],"ImmutableArrayT":["Hello","World"],"IImmutableListT":["Hello","World"],"IImmutableStackT":["World","Hello"],"IImmutableQueueT":["Hello","World"],"IImmutableSetT":["Hello","World"],"ImmutableHashSetT":["Hello","World"],"ImmutableListT":["Hello","World"],"ImmutableStackT":["World","Hello"],"ImmutableQueueT":["Hello","World"],"ImmutableSortedSetT":["Hello","World"],"Object":{}}
Stack Trace:
D:\src\runtime\src\libraries\System.Text.Json\tests\System.Text.Json.Tests\Serialization\PolymorphicTests.cs(376,0): at System.Text.Json.Serialization.Tests.PolymorphicTests.<NestedObjectAsRootObject>g__Verify|9_0(String json)
D:\src\runtime\src\libraries\System.Text.Json\tests\System.Text.Json.Tests\Serialization\PolymorphicTests.cs(393,0): at System.Text.Json.Serialization.Tests.PolymorphicTests.NestedObjectAsRootObject()
also in another run I've seen those 2 happen in a single run which was rather unusual and I think the stream was actually corrupted (somehow ended up with first byte being zero) - interestingly this only affected async related cases and same test using non-async code path was fine:
System.Text.Json.Serialization.Tests.CollectionTestsDynamic_AsyncStreamWithSmallBuffer.ReadAsyncEnumerableOfAsyncEnumerables [FAIL]
System.Text.Json.JsonException : '0x00' is an invalid start of a value. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
---- System.Text.Json.JsonReaderException : '0x00' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
Stack Trace:
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\ThrowHelper.Serialization.cs(265,0): at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonConverterOfT.ReadCore.cs(79,0): at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Read.Helpers.cs(17,0): at System.Text.Json.JsonSerializer.ReadCore[TValue](JsonConverter jsonConverter, Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Read.Stream.cs(574,0): at System.Text.Json.JsonSerializer.ReadCore[TValue](JsonReaderState& readerState, Boolean isFinalBlock, ReadOnlySpan`1 buffer, JsonSerializerOptions options, ReadStack& state, JsonConverter converterBase)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Read.Stream.cs(502,0): at System.Text.Json.JsonSerializer.ContinueDeserialize[TValue](ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, JsonConverter converter, JsonSerializerOptions options)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Read.Stream.cs(358,0): at System.Text.Json.JsonSerializer.ReadAllAsync[TValue](Stream utf8Json, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
D:\src\runtime\src\libraries\System.Text.Json\tests\System.Text.Json.Tests\Serialization\JsonSerializerWrapper.Reflection.cs(157,0): at System.Text.Json.Serialization.Tests.JsonSerializerWrapper.AsyncStreamSerializerWrapper.DeserializeWrapper[T](Stream utf8Json, JsonSerializerOptions options)
D:\src\runtime\src\libraries\System.Text.Json\tests\Common\CollectionTests\CollectionTests.AsyncEnumerable.cs(283,0): at System.Text.Json.Serialization.Tests.CollectionTests.ReadAsyncEnumerableOfAsyncEnumerables()
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\ThrowHelper.cs(279,0): at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Reader\Utf8JsonReader.cs(1111,0): at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Reader\Utf8JsonReader.cs(979,0): at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Reader\Utf8JsonReader.cs(894,0): at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Reader\Utf8JsonReader.cs(272,0): at System.Text.Json.Utf8JsonReader.Read()
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonConverterOfT.ReadCore.cs(27,0): at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
System.Text.Json.Serialization.Tests.CollectionTestsDynamic_AsyncStream.ReadAsyncEnumerableOfAsyncEnumerables [FAIL]
System.Text.Json.JsonException : '0x00' is an invalid start of a value. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
---- System.Text.Json.JsonReaderException : '0x00' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
Stack Trace:
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\ThrowHelper.Serialization.cs(265,0): at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonConverterOfT.ReadCore.cs(79,0): at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Read.Helpers.cs(17,0): at System.Text.Json.JsonSerializer.ReadCore[TValue](JsonConverter jsonConverter, Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Read.Stream.cs(574,0): at System.Text.Json.JsonSerializer.ReadCore[TValue](JsonReaderState& readerState, Boolean isFinalBlock, ReadOnlySpan`1 buffer, JsonSerializerOptions options, ReadStack& state, JsonConverter converterBase)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Read.Stream.cs(502,0): at System.Text.Json.JsonSerializer.ContinueDeserialize[TValue](ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, JsonConverter converter, JsonSerializerOptions options)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonSerializer.Read.Stream.cs(358,0): at System.Text.Json.JsonSerializer.ReadAllAsync[TValue](Stream utf8Json, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
D:\src\runtime\src\libraries\System.Text.Json\tests\System.Text.Json.Tests\Serialization\JsonSerializerWrapper.Reflection.cs(157,0): at System.Text.Json.Serialization.Tests.JsonSerializerWrapper.AsyncStreamSerializerWrapper.DeserializeWrapper[T](Stream utf8Json, JsonSerializerOptions options)
D:\src\runtime\src\libraries\System.Text.Json\tests\Common\CollectionTests\CollectionTests.AsyncEnumerable.cs(283,0): at System.Text.Json.Serialization.Tests.CollectionTests.ReadAsyncEnumerableOfAsyncEnumerables()
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\ThrowHelper.cs(279,0): at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Reader\Utf8JsonReader.cs(1111,0): at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Reader\Utf8JsonReader.cs(979,0): at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first)
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Reader\Utf8JsonReader.cs(894,0): at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Reader\Utf8JsonReader.cs(272,0): at System.Text.Json.Utf8JsonReader.Read()
D:\src\runtime\src\libraries\System.Text.Json\src\System\Text\Json\Serialization\JsonConverterOfT.ReadCore.cs(27,0): at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
Finished: System.Text.Json.Tests
I'll be replacing Assert.Contains in our tests into something else which can possibly provide us some more details
be replacing Assert.Contains in our tests into something else which can possibly provide us some more details
@krwq perhaps you can put it in a shared place - AssertExtensions, we already did something very similar for Equals
https://github.com/dotnet/runtime/pull/67586/files#diff-9f9dc769fe4cfd311f1e57c7572eaeeb09c0c72ffe633131ed0fd8b3ed024994R505
If we build up several of these improved assertions and they show value, we could try to see whether xunit would be interested in any changes.
FWIW the AssertExtensions.Contains already exists and does exactly what I planned to add (which is doing Ordinal comparison by default). I'm currently running tests locally to see if StringComparison.CurrentCulture vs Ordinal make any difference (I want to make sure the failures above are not related to some globalization bug). I'm basically doing CurrentCulture comparison and rather than throwing on "not found" I'm also comparing using Ordinal and if I see any mismatch that will suggest to me this is likely some kind of globalization initialization error. If both checks fail it's likely memory corruption. I'll update once I get a repro (although it might take a day or two depending on the luck)
Ok, finally got lucky - random Assert.Contains failed where actualString.IndexOf(expectedSubstring, StringComparison.CurrentCulture) returned -1 but actualString.IndexOf(expectedSubstring, StringComparison.Ordinal) returned non-negative value.
IMO some issues we're seeing are some kind of globalization issue - maybe cultures didn't get initialized correctly when it happens on multiple threads. We'll likely need to try to write some standalone repro and try to attach debugger to see if we can see what exactly happened and then maybe we can guess what should be changed.
From JSON perspective we likely should change to ordinal compare anyway but it would be good to first nail down what the problem with CurrentCulture is
I'll locally do Ordinal compare for now to flush out other flaky issues and meantime I'm checking with @tarekgh if he's seen anything similar happening elsewhere...
Trying to figure out whether it's Mac specific, Kusto isn't cooperating well, but this query suggests two failure modes: on Windows where lots of nulls are found in the buffer, and on Mac where there is simply an unexpected char repeated after a certain point?
TestResults
| join kind=inner WorkItems on WorkItemId
| join kind=inner Jobs on JobId
//| where Method == "System.Text.Json.SourceGeneration.Tests.SerializationContextTests.SerializeObjectArray_WithCustomOptions"
//| where Type == "LibraryImportGenerator.UnitTests.AdditionalAttributesOnStub"
| where WorkItemFriendlyName == "System.Text.Json.Tests"
//| where WorkItemId == "2558ff82-6a0d-4380-886c-136d7b2e027b"
//| where JobName == "bbb92582-79de-4d65-84ad-c151b994a502"
//| where Method == "SerializeObjectArray_WithCustomOptions"
| where Finished >= now(-30d)
| where Result == "Fail"
| where Message startswith "Assert.Equal() Failure" and Message contains "···"
//| where QueueName == "windows.11.amd64.clientpre.open"
//| where Method == "Count_Timeout_ThrowsAfterTooLongExecution"
//| where Message startswith "System.ArgumentNullException : Value cannot be null. (Parameter 'package"
| project Type, Method,
Pipeline = tostring(parse_json(Properties).DefinitionName),//WorkItemFriendlyName,
Pipeline_Configuration = tostring(parse_json(Properties).configuration),
OS = QueueName,
Arch = tostring(parse_json(Properties).architecture),
// Test = Type1,
//Result,
Finished,
Build,//, WorkItemFriendlyName, WorkItemName,
//Duration,
// Method,
// Build = tostring(parse_json(Properties).BuildNumber),
Message//,//
,StackTrace
| Type | Method | Pipeline | Pipeline_Configuration | OS | Arch | Finished | Build | Message | StackTrace |
|---|---|---|---|---|---|---|---|---|---|
| System.Text.Json.Tests.Utf8JsonWriterTests | Writing3MBBase64Bytes | corefx-ci | Release | windows.7.amd64.open.svc | x64 | 2022-03-12 00:27:48.0730000 | Assert.Equal() Failure\r\n ↓ (pos 2424184)\r\nExpected: ···qKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq···\r\nActual: ···qKioqKioqKioqKioqKio\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0qKioqKioqKioqKio\0\0\0\0\0\0\0\0\0···\r\n ↑ (pos 2424184) | at System.Text.Json.JsonTestHelper.AssertContents(String expectedValue, ArrayBufferWriter`1 buffer) in //src/System.Text.Json/tests/JsonTestHelper.cs:line 718 at System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes(Boolean formatted, Boolean skipValidation) in //src/System.Text.Json/tests/Utf8JsonWriterTests.cs:line 2746 |
|
| System.Text.Json.Tests.Utf8JsonWriterTests | Writing3MBBase64Bytes | corefx-official | Release | windows.7.amd64.svc | x64 | 2022-03-14 21:33:09.7100000 | Assert.Equal() Failure\r\n ↓ (pos 3769341)\r\nExpected: ···qKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq···\r\nActual: ···qKioqKioqKioqKioqKio\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0qKioqKioqKioqKio\0\0\0\0\0\0\0\0\0···\r\n ↑ (pos 3769341) | at System.Text.Json.JsonTestHelper.AssertContents(String expectedValue, ArrayBufferWriter`1 buffer) in //src/System.Text.Json/tests/JsonTestHelper.cs:line 718 at System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes(Boolean formatted, Boolean skipValidation) in //src/System.Text.Json/tests/Utf8JsonWriterTests.cs:line 2737 |
|
| System.Text.Json.Serialization.Tests.StreamTests_Sync | HandleCollectionsAsync<ImmutableDictionary<String, String>, String> | runtime-extra-platforms | Release | osx.1200.amd64.open | x64 | 2022-03-31 02:05:54.8690000 | Assert.Equal() Failure ↓ (pos 6998) Expected: ···eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee··· Actual: ···eeeeeeeeeeeeeeeeeeeeddddddddddddddddddddddddddddddddddddddddd··· ↑ (pos 6998) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 61 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 45 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 50 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 30 --- End of stack trace from previous location --- |
|
| System.Text.Json.Serialization.Tests.StreamTests_Sync | HandleCollectionsAsync<ConcurrentQueue<StreamTests+ImmutableStructWithStrings>, StreamTests+ImmutableStructWithStrings> | runtime-extra-platforms | Release | osx.1200.amd64.open | x64 | 2022-03-31 10:14:04.9450000 | Assert.Equal() Failure ↓ (pos 5610) Expected: ···ccccccccccccccccccccddddddddddddddddddddddddddddddddddddddddd··· Actual: ···ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc··· ↑ (pos 5610) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 61 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 45 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 55 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 50 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in /_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 30 --- End of stack trace from previous location --- |
|
| System.Text.Json.Serialization.Tests.StreamTests_Sync | HandleCollectionsAsync<GenericICollectionWrapper<String>, String> | runtime-extra-platforms | Release | osx.1200.amd64.open | x64 | 2022-03-31 18:34:57.1600000 | Assert.Equal() Failure ↓ (pos 14482) Expected: ···kkkkkkkkkkkkkkkkkkkkooooooooooooooooooooooooooooooooooooooooo··· Actual: ···kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk··· ↑ (pos 14482) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 61 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 55 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 50 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 30 --- End of stack trace from previous location --- |
|
| System.Text.Json.Serialization.Tests.StreamTests_Async | HandleCollectionsAsync<Stack<String>, String> | runtime | Release | osx.1200.amd64.open | x64 | 2022-04-01 05:28:55.5400000 | Assert.Equal() Failure ↓ (pos 14598) Expected: ···oooooooooooooooooooorrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr··· Actual: ···ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo··· ↑ (pos 14598) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 61 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 55 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 54 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 30 --- End of stack trace from previous location --- |
|
| System.Text.Json.Serialization.Tests.StreamTests_Sync | HandleCollectionsAsync<ImmutableStack<StreamTests+ImmutableStructWithStrings>, StreamTests+ImmutableStructWithStrings> | runtime-extra-platforms | Release | osx.1200.amd64.open | x64 | 2022-04-01 21:49:51.7660000 | Assert.Equal() Failure ↓ (pos 7474) Expected: ···iiiiiiiiiiiiiiiiiiiixxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx··· Actual: ···iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii··· ↑ (pos 7474) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 61 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 45 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 55 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 54 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in /_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 33 --- End of stack trace from previous location --- |
|
| System.Text.Json.Serialization.Tests.StreamTests_Sync | HandleCollectionsAsync<ImmutableStack<StreamTests+ClassWithKVP>, StreamTests+ClassWithKVP> | runtime | Release | osx.1200.amd64.open | x64 | 2022-04-03 08:50:53.7770000 | Assert.Equal() Failure ↓ (pos 13390) Expected: ···xxxxxxxxxxxxxxxxxxxxooooooooooooooooooooooooooooooooooooooooo··· Actual: ···xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx··· ↑ (pos 13390) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 61 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 45 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 45 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 55 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 50 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 30 --- End of stack trace from previous location --- |
|
| System.Text.Json.Serialization.Tests.StreamTests_Sync | HandleCollectionsAsync<Collection<StreamTests+ImmutableStructWithStrings>, StreamTests+ImmutableStructWithStrings> | runtime | Release | osx.1200.amd64.open | x64 | 2022-04-03 08:50:53.7770000 | Assert.Equal() Failure ↓ (pos 8986) Expected: ···ppppppppppppppppppppwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww··· Actual: ···ppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp··· ↑ (pos 8986) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 61 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 45 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 55 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 45 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 50 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 33 --- End of stack trace from previous location --- |
|
| System.Text.Json.Serialization.Tests.StreamTests_Sync | HandleCollectionsAsync<GenericICollectionWrapper<StreamTests+ImmutableStructWithStrings>, StreamTests+ImmutableStructWithStrings> | runtime-extra-platforms | Release | osx.1200.amd64.open | x64 | 2022-04-04 21:56:37.7790000 | Assert.Equal() Failure ↓ (pos 6566) Expected: ···ttttttttttttttttttttsssssssssssssssssssssssssssssssssssssssss··· Actual: ···ttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttt··· ↑ (pos 6566) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 73 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 73 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 73 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 54 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in /_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 30 --- End of stack trace from previous location --- |
|
| System.Text.Json.Serialization.Tests.StreamTests_AsyncWithSmallBuffer | HandleCollectionsAsync<ConcurrentStack<String>, String> | runtime-extra-platforms | Release | osx.1200.amd64.open | x64 | 2022-04-06 19:25:17.9870000 | Assert.Equal() Failure ↓ (pos 10422) Expected: ···sssssssssssssssssssslllllllllllllllllllllllllllllllllllllllll··· Actual: ···sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss··· ↑ (pos 10422) |
at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 61 at System.Text.Json.JsonTestHelper.AssertJsonEqual(JsonElement expected, JsonElement actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 55 at System.Text.Json.JsonTestHelper.AssertJsonEqual(String expected, String actual) in //src/libraries/System.Text.Json/tests/Common/JsonTestHelper.cs:line 20 at System.Text.Json.Serialization.Tests.StreamTests.TestDeserialization[TCollection,TElement](String json, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 87 at System.Text.Json.Serialization.Tests.StreamTests.PerformSerialization[TCollection,TElement](TCollection collection, JsonSerializerOptions options) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 54 at System.Text.Json.Serialization.Tests.StreamTests.HandleCollectionsAsync[TCollection,TElement](TCollection collection, Int32 bufferSize, Witness`1 elementType) in //src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.Collections.cs:line 30 --- End of stack trace from previous location --- |
Ok, finally got lucky - random Assert.Contains failed where actualString.IndexOf(expectedSubstring, StringComparison.CurrentCulture) returned -1 but actualString.IndexOf(expectedSubstring, StringComparison.Ordinal) returned non-negative value.
This is possible to happen and can be legitimate. With the cultural search there can be some clusters (multiple consequent characters) which you cannot break inside, and the search cannot stop in the middle of this cluster. In the other hand ordinal search work per character which will not have this case. Send me the text you have used in such case, and I can try to find out this specific case.
By the way, we had a similar case in .NET 5.0 when searching for \n in \r\n. We had to work around this specific case in .NET 6.0 because this was a major case many users were complaining about.
@tarekgh 2 example texts can be seen in the stack traces here: https://github.com/dotnet/runtime/issues/59678#issuecomment-1090626404 - the only thing is that it happens only once per many runs of JSON tests (it might be anything between 0 up to 24h, the 3 cases linked here were around 8h of running all tests in a loop each)
@krwq can you copy and paste the strings here? I have hard time to see them from https://github.com/dotnet/runtime/issues/59678#issuecomment-1090626404.
What @danmoseley mentioned in the comment https://github.com/dotnet/runtime/issues/59678#issuecomment-1093067266 suggest there could be some memory corruption happening somewhere which can cause the result you are seeing. Also, do we know what current culture is used when running the tests? Is there a way we can get a memory dump when the issue occurs?
I've looked at the recent failures and the failures seem much different than the original reports. Now seems that majority of the issues is happening in large JSONs and many (but not all) are related to somehow swapping empty string with null. Another observation is that if it fails in a single run it will likely fail many test cases. We should keep observing this for any potential patterns. For now this doesn't seem actionable. We should try to see if we can repro locally and perhaps attach debugger and search for clues where the culprit might be
Even if this has morphed from the original issue, we should take a look at this during 9.0.0 since it causes CI failures.
I think there are other better checks that are more specific that are catching the things being caught there now like #100111 and #98578 so I'm closing this
@lewing / @bartonjs this is our top reported known issue right now, I think that the search text is too board.
System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers
To Larry's point above, is there a more specific search text for the failure that this issue is tracking?
Yeah I'm closing it again and I'll make mroe specific tests if any aren't caught by the others
the problem is there are failures of this specific test and the failures look like:
/datadisks/disk1/work/9B2B0894/w/B43409F0/e /datadisks/disk1/work/9B2B0894/w/B43409F0/e
Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Text.Json.Tests (found 7405 of 7466 test cases)
Starting: System.Text.Json.Tests (parallel test collections = on [2 threads], stop on fail = off)
System.Text.Json.Tests.Utf8JsonReaderTests.TestDepth(depth: 256) [SKIP]
Takes very long to run on interpreter.
System.Text.Json.Tests.Utf8JsonReaderTests.TestDepth(depth: 512) [SKIP]
Takes very long to run on interpreter.
System.Text.Json.Tests: [Long Running Test] 'System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes', Elapsed: 00:05:42
System.Text.Json.Tests: [Long Running Test] 'System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers', Elapsed: 00:02:01
System.Text.Json.Tests: [Long Running Test] 'System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers', Elapsed: 00:04:01
Killed
['System.Text.Json.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]
https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-101117-merge-608176b4786749e2b4/System.Text.Json.Tests/1/console.6c31de45.log?helixlogtype=result which isn't easy to distinguish with build analysis