Dahomey.Cbor icon indicating copy to clipboard operation
Dahomey.Cbor copied to clipboard

Library is not threadsafe

Open AngelicosPhosphoros opened this issue 3 years ago • 4 comments

If code with serialization and deserialization runs in multiple threads, it doesn't serialize and deserialize objects properly.

Example code:

using Dahomey.Cbor;
using System;
using System.Buffers;
using System.Collections.Generic;
using System.Linq;
using System.Threading;

namespace TestCborError
{
    readonly struct SomeStrings
    {
        public readonly string F1;
        public readonly string F2;
        public readonly string F3;
        public readonly string F4;

        public SomeStrings(string f1, string f2, string f3, string f4)
        {
            F1 = f1;
            F2 = f2;
            F3 = f3;
            F4 = f4;
        }
        public bool Equals(in SomeStrings strings)
        {
            return F1 == strings.F1 &&
                   F2 == strings.F2 &&
                   F3 == strings.F3 &&
                   F4 == strings.F4;
        }
        public override bool Equals(object obj)
        {
            return obj is SomeStrings strings && Equals(strings);
        }
    }
    class WithInnerList
    {
        public List<SomeStrings> InnerLists { get; set; } = new List<SomeStrings>();

        public override bool Equals(object obj)
        {
            return obj is WithInnerList list &&
                   Enumerable.SequenceEqual(InnerLists, list.InnerLists);
        }
    }

    class Program
    {
        static byte[] Serialized;
        static void InitStatic()
        {
            var original = MakeObject();
            ArrayBufferWriter<byte> buffer = new();
            Cbor.Serialize(original, buffer);
            Serialized = buffer.WrittenSpan.ToArray();
        }
        static void Main(string[] args)
        {
            int num_threads = int.Parse(args[0]);
            if (args.Length > 1)
            {
                InitStatic();
            }
            List<Thread> threads = new();
            for (int i = 0; i < num_threads; ++i)
            {
                Thread t = new(DoWork);
                threads.Add(t);
                t.Start();
            }
            foreach (var t in threads)
            {
                t.Join();
            }
        }

        static void DoWork()
        {
            var original = MakeObject();
            for (int i = 0; i < 1000; ++i)
            {
                ArrayBufferWriter<byte> buffer = new();
                Cbor.Serialize(original, buffer);
                ReadOnlySpan<byte> local_serialized = buffer.WrittenSpan;
                if (Serialized!=null && !local_serialized.SequenceEqual(Serialized))
                {
                    Console.WriteLine($"Serialized not equal in {Thread.CurrentThread.ManagedThreadId}");
                    continue;
                }
                var restored = Cbor.Deserialize<WithInnerList>(buffer.WrittenSpan);
                if (!Equals(original, restored))
                {
                    Console.WriteLine($"Deserialized is not equal in {Thread.CurrentThread.ManagedThreadId}");
                }
            }
        }

        static WithInnerList MakeObject()
        {
            return new()
            {
                InnerLists = new()
                {
                    new SomeStrings("A", "B", "C", "D"),
                    new SomeStrings("E", "F", "G", "H"),
                    new SomeStrings("I", "J", "K", "L"),
                    new SomeStrings("M", "N", "R", "S"),
                }
            };
        }
    }
}

So, executions of this program depends on inputs: .\TestCborError.exe 1 -> Clear output .\TestCborError.exe 2 -> Many lines of Deserialized is not equal in 4 and Deserialized is not equal in 5 .\TestCborError.exe 2 --init-static -> Clear output.

It looks like that some part of initialization of Cbor class is not threadsafe and if it runs in many threads, there are invalid serializations and deserializations.

My setup:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Dahomey.Cbor" Version="1.15.3" />
  </ItemGroup>

</Project>

and I run this code in Windows 10.

AngelicosPhosphoros avatar Sep 05 '21 22:09 AngelicosPhosphoros

Same problem on our end

ondrejtomcik avatar Jun 27 '22 08:06 ondrejtomcik

@mcatanzariti Have you had a look at this?

rmja avatar Oct 25 '22 06:10 rmja

Yes, I could repro it but the fix is not simple. I'm going to work on it

mcatanzariti avatar Oct 25 '22 09:10 mcatanzariti

reproduced

mcatanzariti avatar Oct 25 '22 12:10 mcatanzariti