LiteDB icon indicating copy to clipboard operation
LiteDB copied to clipboard

[QUESTION] Object synchronization method was called from an unsynchronized block of code

Open jokker23 opened this issue 4 years ago • 42 comments

I'm getting this error intermittently while running with ConnectionType.Shared and don't with the default.

System.ApplicationException: Object synchronization method was called from an unsynchronized block of code.

   at System.Threading.Mutex.ReleaseMutex()
   at LiteDB.SharedEngine.CloseDatabase()
   at LiteDB.SharedEngine.Upsert(String collection, IEnumerable`1 docs, BsonAutoId autoId)
System.ApplicationException: Object synchronization method was called from an unsynchronized block of code.
   at System.Threading.Mutex.ReleaseMutex()
   at LiteDB.SharedEngine.CloseDatabase()
   at LiteDB.SharedDataReader.Dispose()
   at LiteDB.LiteQueryable`1.ToDocuments()+MoveNext()
   at System.Linq.Enumerable.SelectEnumerableIterator`2.ToList()

I tried researching this but haven't come up with any ideas. Does anyone have any suggestions on why I'm getting this?

jokker23 avatar Mar 09 '20 15:03 jokker23

@jokker23 Could you please provide us with more information like, which version of LiteDB you're using, OS, net runtime, et cetera 🙂

JensSchadron avatar Mar 09 '20 23:03 JensSchadron

Certainly,

LiteDB: v5.0.2 Win10 .NET Core 3.1

Also using Newtonsoft.Json 12.0.3 RabbitMQ.Client 5.1.1 Microsoft.Extensions.Caching.Memory 3.1.1 Microsoft.Extensions.Configuration.Abstractions 3.1.1 Microsoft.Extensions.Configuration.Json 3.1.1 Microsoft.Extensions.DependencyInjection 3.1.1 Microsoft.Extensions.Logging 3.1.1 Microsoft.Extensions.Logging.Console 3.1.1 NLog.Extensions.Logging 1.6.1 And a custom SDK dll

Is there other information that is interesting?

jokker23 avatar Mar 10 '20 13:03 jokker23

@jokker23 Could you provide a sample code that we may use to replicate the issue?

lbnascimento avatar Mar 10 '20 19:03 lbnascimento

@jokker23 Could you test with the current master? The transaction model was entirely rewritten.

lbnascimento avatar Mar 12 '20 20:03 lbnascimento

Yes sorry I will try to accommodate both requests as soon as possible. I'm currently on overload making VPN tunnels for home workers, sorry for the delay.

jokker23 avatar Mar 12 '20 20:03 jokker23

I got the same error, too.

System.ApplicationException:  Object synchronization method was called from an unsynchronized block of code
於 System.Threading.Mutex.ReleaseMutex()
於 LiteDB.SharedEngine.CloseDatabase()
於 LiteDB.SharedDataReader.Dispose(Boolean disposing)
於 LiteDB.SharedDataReader.Dispose()
於 LiteDB.LiteQueryable`1.<ToDocuments>d__26.MoveNext()
於 System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
於 System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
於 System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)

alonstar avatar Mar 23 '20 09:03 alonstar

@alonstar What version are you running?

lbnascimento avatar Mar 23 '20 13:03 lbnascimento

@alonstar What version are you running?

5.0.4 and .net framework 4.7.2

alonstar avatar Mar 23 '20 15:03 alonstar

I've the same problem with 5.0.5 on .Net core 3.1 when multiple API calls are trying to connect to the same database.

Update 1: In my application I use Autofac and configured it to create only a single instance of the LiteDB database (wrapped in a provider, so actually the provider was created once and the internal LiteDB as well). The connectionstring contains 'Connection=shared'. In this situation I get the same error as described above.

I've changed my Autofac configuration so it will always create a new provider (with internally a new LiteDb instance). This solved this specific problem, but now I'm running into file in use problems?

Update 2: Well I now reverted the my Autofac configuration into what I initially had. So a single instance of the provider, with internally a single instance of LiteDB. Changed the connectionstring from 'Connection=shared' to 'Connection=direct' and this seems to be stable after a couple of tests.

mwlpeeters avatar Apr 03 '20 13:04 mwlpeeters

I can reliably reproduce this exception (not necessarily same issue) with the following test ran using "Debug test" mode (RMB \ Debug). "Run test" seems to be usually working. If it is still not reproducible, try to change the number in Sleep(50).

[Fact]
public void Just_DataStream3()
{
    using var mem = new MemoryStream();
    var settings = new EngineSettings()
    {
        DataStream = mem,
        Filename = "Just_DataStream3" //! see https://github.com/mbdavid/LiteDB/issues/1614
    };
    using var engine = new SharedEngine(settings);

    void worker()
    {
        using var db = new LiteDatabase(engine, null, false);
        var col = db.GetCollection("test");
        for (int i = 0; i < 100; ++i)
        {
            col.Insert(new BsonDocument() { ["i"] = i });
            System.Threading.Thread.Sleep(50);
        }
    }

    var thread1 = new System.Threading.Thread(worker);
    var thread2 = new System.Threading.Thread(worker);
    var thread3 = new System.Threading.Thread(worker);
    thread1.Start();
    thread2.Start();
    thread3.Start();
    thread1.Join();
    thread2.Join();
    thread3.Join();

    using var db = new LiteDatabase(engine, null, false);
    var col = db.GetCollection("test");
    Assert.Equal(300, col.Count());
}

nightroman avatar Apr 07 '20 03:04 nightroman

Can you repro the problem with the above code? Please let me know. If not then I will continue the search for repro cases. I think problems might be related to https://github.com/mbdavid/LiteDB/issues/1614 which is not well repro, too.

nightroman avatar Apr 08 '20 05:04 nightroman

@nightroman Yes, I was able to reproduce the problem.

lbnascimento avatar Apr 08 '20 15:04 lbnascimento

Any updates? Having the same issue :(

Sn3b avatar May 04 '20 09:05 Sn3b

Is there any progress regarding this issue or at least an idea what is causing the issue? I'm getting this issue a lot and it is breaking the application. I will need to look for alternatives as soon as possible if this can't be fixed so if you can please tell me what is the situation. Thank you

temnava avatar May 13 '20 13:05 temnava

@jokker23 @alonstar @mwlpeeters @nightroman @Sn3b @temnava What happens is the following: Mutex keeps track of which thread holds the lock and only allows that specific thread to release the lock. Releasing the lock from a different thread would not be an issue in our case, since we only use Mutex for process synchronization - thread synchronization is done with other mecanisms. However, Mutex prevents us from doing that.

I tried replacing Mutex with Semaphore, and it completely fixed this issue. However, it created an issue of its own: if the process that holds the lock is abruptly closed without releasing it, all other processes are locked out indefinitely. This does not happen with Mutex because it throws an AbandonedMutexException, which allows us to detect when it happens.

None of the solutions is perfect, and I'm not sure which is preferable, but I'm tempted to keep the Shared mode as it currently is. It will be completely reworked for v5.1 anyway.

lbnascimento avatar May 13 '20 15:05 lbnascimento

Hello, than you very much for a fast response and this great library. Can you provide the code somewhere for the solution with Semaphore implementation so that I can use it untill the 5.1 is released? Btw is there any ETA for 5.1 and will this issues be possible in that new implementation? Regards

temnava avatar May 13 '20 15:05 temnava

@temnava Here's a pastebin with the code that uses Semaphore: https://pastebin.com/yAmzvrWY. Just replace the entire SharedEngine.cs with this code.

lbnascimento avatar May 13 '20 16:05 lbnascimento

@lbnascimento From the front end I had 2 calls that called different methods in a controller, 1 to add an item and 1 to update another one. In the end I just merged my 2 calls into 1 and created a new method in my controller so that the operations didn't happen simutanously.

Edit: I use a connection string with Connection=shared because I also have another service that needs to access that same DB.

Sn3b avatar May 13 '20 16:05 Sn3b

So if i got this correct, there is a problem with concurrent access when multiple threads do operations on the database, because the the mutex must be released by the thread which took the lock, and the lock is opened when the count is 1 and closed when it goes back to 0, meaning the "last" thread active closes it.

So imo the mutex is useless, as it doesn't make the application threadsafe. Currently the user is responsible for ensuring there is no concurrent access, making the mutex excessive.

So in this case I think it's quite obvious that the advantages of the semaphore (actually being threadsafe) outweigh the advantage of knowing when a mutex is abandoned massively.

*Edit: I think it is possible to do this with just "lock" statements, i'll try to create a PR for this. Advantage is that even when something goes wrong, you won't keep the lock.

*Edit 2: I didn't realize the mutexes where used to synchronize across processes, I thought it was for synchronizing across threads only, and other mechanisms were in place to synchronize across processes (i.e. filechange counter such as SQLite)

MidasLamb avatar May 27 '20 12:05 MidasLamb

@MidasLamb Mutex is only used in the SharedEngine, for synchronization between processes. The LiteEngine uses lock statements for thread synchronization.

The SharedEngine will be reworked for future versions and will no longer use Mutex.

lbnascimento avatar May 27 '20 14:05 lbnascimento

@lbnascimento I noticed that when I was creating a PR. Do you have an ETA for this change? Currently we're just surrounding all the accesses with a lock statement to prevent the error from being thrown.

MidasLamb avatar May 27 '20 14:05 MidasLamb

Hello, is there any progress in resolving this issue? I can see some commits in the "single_file" branch two months ago that might be fixes for this issue, a I right?

temnava avatar Aug 10 '20 21:08 temnava

I've faced the same issue with a lot of concurrent calls when using Shared connection mode. Any updates?

dropsonic avatar Aug 13 '20 14:08 dropsonic

I'm hitting this on changing to shared connection in various places but seems in my data access layer code, using a lock around the code as suggested above and returning concrete lists with ToList for IEnumerable results seems to help. It means the DB read occurs at that point, not in subsequent LINQ queries.

I'm reviewing using shared connection though and thinking of other ways with multi-process, aside from a full database (some issues blocking that with some customer requirements and limits on installed database products, especially if they have to maintain them). My initial query on #1773 maybe is answered by a server based LiteDB though, if it can be embedded in some way.

tjmoore avatar Oct 09 '20 20:10 tjmoore

I'm getting the same error with Shared Engine.

System.ApplicationException: Object synchronization method was called from an unsynchronized block of code.

And then, I tried to patch it, debugging for two weeks. It's working fine for me now. Unfortunately, "dot Net Standard 1.3" does not support reflection. That is the reason why it looks so ugly.

LiteDB/Client/Shared/SharedEngine.cs

using LiteDB.Engine;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Linq.Expressions;
using System.Threading;
#if NETFRAMEWORK
using System.Security.AccessControl;
using System.Security.Principal;
#endif

namespace LiteDB
{
    public class SharedEngine : ILiteEngine
    {
        private readonly EngineSettings _settings;
        private readonly Mutex _mutex;
        private LiteEngine _engine;
        private Random rand = new Random(
                    Guid.NewGuid().GetHashCode()
                    );

        //Thread safe bool, default is 0 as false;
        private volatile int _threadSafeBoolBackValueForIsMutexLocking = 0;
        private bool isMutexLocking
        {
            get
            {
                return (Interlocked.CompareExchange(ref _threadSafeBoolBackValueForIsMutexLocking, 1, 1) == 1);
            }
            set
            {
                if (value) Interlocked.CompareExchange(ref _threadSafeBoolBackValueForIsMutexLocking, 1, 0);
                else Interlocked.CompareExchange(ref _threadSafeBoolBackValueForIsMutexLocking, 0, 1);
            }
        }

        //Thread safe bool, default is 0 as false;
        private volatile int _threadSafeBoolBackValueForIsDisposed = 0;
        private bool isDisposed
        {
            get
            {
                return (Interlocked.CompareExchange(ref _threadSafeBoolBackValueForIsDisposed, 1, 1) == 1);
            }
            set
            {
                if (value) Interlocked.CompareExchange(ref _threadSafeBoolBackValueForIsDisposed, 1, 0);
                else Interlocked.CompareExchange(ref _threadSafeBoolBackValueForIsDisposed, 0, 1);
            }
        }


        public SharedEngine(EngineSettings settings)
        {
            _settings = settings;

            var name = Path.GetFullPath(settings.Filename).ToLower().Sha1();

            try
            {
#if NETFRAMEWORK
                var allowEveryoneRule = new MutexAccessRule(
                    new SecurityIdentifier(WellKnownSidType.WorldSid, null),
                    MutexRights.FullControl, AccessControlType.Allow);

                var securitySettings = new MutexSecurity();
                securitySettings.AddAccessRule(allowEveryoneRule);

                _mutex = new Mutex(false, "Global\\" + name + ".Mutex", out _, securitySettings);
#else
                _mutex = new Mutex(false, "Global\\" + name + ".Mutex");
#endif
            }
            catch (NotSupportedException ex)
            {
                throw new PlatformNotSupportedException
                    ("Shared mode is not supported in platforms that do not implement named mutex.", ex);
            }
        }

        /// <summary>
        /// Open database in safe mode
        /// </summary>
        private void OpenDatabase()
        {
            if (!isMutexLocking)
            {
                lock (_mutex)
                {
                    isMutexLocking = true;
                    if (_engine == null)
                    {
                        try
                        {
                            _mutex.WaitOne();
                        }
                        catch (AbandonedMutexException) { }
                        catch { isMutexLocking = false; throw; }

                        try
                        {
                            _engine = new LiteEngine(_settings);
                            isDisposed = false;
                        }
                        catch
                        {
                            try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                            isMutexLocking = false;
                            throw;
                        }
                    }
                    isMutexLocking = false;
                }
            }
            else
            {
                if (_engine == null)
                {
                    try
                    {
                        _engine = new LiteEngine(_settings);
                        isDisposed = false;
                    }
                    catch
                    {
                        try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                        isMutexLocking = false;
                        throw;
                    }
                }
            }
        }

        /// <summary>
        /// Dequeue stack and dispose database on empty stack
        /// </summary>
        private void CloseDatabase()
        {
            if (!isMutexLocking)
            {
                lock (_mutex)
                {
                    isMutexLocking = true;
                    if (_engine != null)
                    {
                        this.Dispose(true);
                    }
                    isMutexLocking = false;
                }
            }
            else
            {
                if (_engine != null)
                {
                    this.Dispose(true);
                }
            }
        }

#region Transaction Operations

        public bool BeginTrans()
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;

                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                bool val = false;
                try
                {
                    var result = _engine.BeginTrans();
                    /*
                    if(result == false)
                    {

                    }
                    */
                    val = result;
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public bool Commit()
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;

                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                if (_engine == null || isDisposed)
                {
                    isDisposed = true;
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    return false;
                }

                bool val = false;
                try
                {
                    val = _engine.Commit();
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public bool Rollback()
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;

                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                if (_engine == null || isDisposed)
                {
                    isDisposed = true;
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    return false;
                }

                bool val = false;
                try
                {
                    val = _engine.Rollback();
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

#endregion

#region Read Operation

        public IBsonDataReader Query(string collection, Query query)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;

                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                SharedDataReader bsonDataReader = null;
                try
                {
                    var reader = _engine.Query(collection, query);
                    bsonDataReader = new SharedDataReader(reader, () => this.Dispose("Query"));
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                if(bsonDataReader != null)
                {
                    return bsonDataReader;
                }
                else
                {
                    throw new NullReferenceException("Nulled SharedDataReader.");
                }
            }
        }

        public BsonValue Pragma(string name)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;

                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                BsonValue val = null;
                try
                {
                    val = _engine.Pragma(name);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public bool Pragma(string name, BsonValue value)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                bool val = false;
                try
                {
                    val = _engine.Pragma(name, value);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

#endregion

#region Write Operations

        public int Checkpoint()
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();
                
                int val;
                try
                {
                    val = _engine.Checkpoint();
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public long Rebuild(RebuildOptions options)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                long val;
                try
                {
                    val = _engine.Rebuild(options);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public int Insert(string collection, IEnumerable<BsonDocument> docs, BsonAutoId autoId)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                int val;
                try
                {
                    val = _engine.Insert(collection, docs, autoId);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public int Update(string collection, IEnumerable<BsonDocument> docs)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                int val;
                try
                {
                    val = _engine.Update(collection, docs);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public int UpdateMany(string collection, BsonExpression extend, BsonExpression predicate)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                int val;
                try
                {
                    val = _engine.UpdateMany(collection, extend, predicate);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();

                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public int Upsert(string collection, IEnumerable<BsonDocument> docs, BsonAutoId autoId)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                int val;
                try
                {
                    val = _engine.Upsert(collection, docs, autoId);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();
                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public int Delete(string collection, IEnumerable<BsonValue> ids)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                int val;
                try
                {
                    val = _engine.Delete(collection, ids);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();
                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public int DeleteMany(string collection, BsonExpression predicate)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                int val;
                try
                {
                    val = _engine.DeleteMany(collection, predicate);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();
                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public bool DropCollection(string name)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                bool val = false;
                try
                {
                    val = _engine.DropCollection(name);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();
                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public bool RenameCollection(string name, string newName)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                bool val = false;
                try
                {
                    val = _engine.RenameCollection(name, newName);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();
                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public bool DropIndex(string collection, string name)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                bool val = false;
                try
                {
                    val = _engine.DropIndex(collection, name);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();
                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }

        public bool EnsureIndex(string collection, string name, BsonExpression expression, bool unique)
        {
            EnsureLocking();
            lock (_mutex)
            {
                isMutexLocking = true;
                try
                {
                    _mutex.WaitOne();
                }
                catch (AbandonedMutexException) { }
                catch { isMutexLocking = false; throw; }

                this.OpenDatabase();

                bool val = false;
                try
                {
                    val = _engine.EnsureIndex(collection, name, expression, unique);
                }
                catch
                {
                    this.CloseDatabase();
                    try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                    isMutexLocking = false;
                    throw;
                }

                this.CloseDatabase();
                try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
                isMutexLocking = false;
                return val;
            }
        }
#endregion

        private void EnsureLocking()
        {
            int count = 0;
            System.Threading.Tasks.Task delay;
            while (isMutexLocking)
            {
                delay = System.Threading.Tasks.Task.Delay(
                    TimeSpan.FromMilliseconds(
                        rand.Next(3, 7)
                        )
                    );
                delay.Wait();
                if (!isMutexLocking) { break; }
                if (delay.IsCompleted)
                {
                    ++count;

                    if (count > 100)
                    {
                        count = 0;
                        throw new TimeoutException("Shared threading controller with Mutex is Locking.");
                    }
                }
            }
        }

        public void Dispose()
        {
            this.Dispose(false);
        }

        public void Dispose(string fromQuery = "yes")
        {
            this.Dispose(true);
            isMutexLocking = false;
            GC.SuppressFinalize(this);
        }

        ~SharedEngine()
        {
            this.Dispose(false);
            try { _mutex.ReleaseMutex(); } catch { isMutexLocking = false; throw; }
            isMutexLocking = false;
            GC.SuppressFinalize(this);
        }

        protected virtual void Dispose(bool disposing)
        {
            if (_engine == null || isDisposed)
            {
                isDisposed = true;
                return;
            }

            if (disposing)
            {
                if (_engine != null)
                {
                    if (!isMutexLocking)
                    {
                        lock (_mutex)
                        {
                            isMutexLocking = true;

                            try
                            {
                                _engine.Dispose();
                                _engine = null;
                            }
                            catch
                            {
                                _engine = null;
                            }
                            finally
                            {
                                isDisposed = true;
                            }

                            isMutexLocking = false;
                        }
                    }
                    else
                    {
                        try
                        {
                            _engine.Dispose();
                            _engine = null;
                        }
                        catch
                        {
                            _engine = null;
                        }
                        finally
                        {
                            isDisposed = true;
                        }
                    }
                }
            }

            GC.Collect();
        }
    }
}

What it needs is using a retry:

/* codes for using Shared Engine LiteDB database and get the collection */
var collection = ... ;

/* using a retry , start from here */
bool isSuccessful = true;
int retryCount = 0;
while(true){
    try
    {
        isSuccessful = isSuccessful && collection.Upsert<T>(entity);
        if (isSuccessful) { break; }
    }
    catch
    {
        ++retryCount;

        /** ... code for doing log for retry count, catched exception, and others ... **/

        if (retryCount > 10) { throw; }
    }
}

I am not getting Exception from throw, and it log retry count less then 2 times with my project code.

Aspen-TWN avatar Dec 29 '20 04:12 Aspen-TWN

@nightroman @MidasLamb @lbnascimento Could you try this (SharedEngine.cs, require "System.Threading.Thread" package for netstandard1.3) : https://pastebin.com/VHXSrjm0

My solution: force all mutex operations to run on the same thread. I found this: https://github.com/matthiaswelz/journeyofcode/blob/master/SingleThreadScheduler/SingleThreadScheduler/SingleThreadTaskScheduler.cs

This test (https://github.com/mbdavid/LiteDB/issues/1546#issuecomment-610147941) is passed.

Note: I forgot the Dispose() method

Thanks

haiduong87 avatar Apr 28 '21 18:04 haiduong87

I got the same error, too.

sjahongir avatar Jun 25 '21 04:06 sjahongir

Same problem here after switching to a shared connection with LiteDb 5, had to ditch LiteDB and switch to our MongoDB layer because this broke our application

Inurias avatar Jun 30 '21 15:06 Inurias

Any news on this?

LuigiMaestrelli avatar Sep 02 '21 19:09 LuigiMaestrelli

Following, as this is an issue for us as well. Any ETA?

Pokis avatar Feb 08 '22 13:02 Pokis