LiteDB [BUG] Intermittent Invalid IndexPage/DataPage buffer on 0

[BUG] Intermittent Invalid IndexPage/DataPage buffer on 0

Open nabeelio opened this issue 4 years ago • 44 comments

Version Which LiteDB version/OS/.NET framework version are you using. (REQUIRED) 5.0.1. Here's the full crash report for a load, and then a crash report for a save: Framework 4.7

LOAD (DataPage error)

Package: Acars
Version: 1.0.0.823
OS: WINDOWS
Windows: 10.0.18363
Date: 2020-02-21T02:52:10.0507405Z

Exception Stack:
LiteDB.LiteException: Invalid DataPage buffer on 0
  at LiteDB.Engine.DataPage..ctor(PageBuffer buffer)
  at LiteDB.Engine.BasePage.ReadPage[T](PageBuffer buffer)
  at LiteDB.Engine.Snapshot.ReadPage[T](UInt32 pageID, FileOrigin& origin, Int64& position, Int32& walVersion)
  at LiteDB.Engine.Snapshot.GetPage[T](UInt32 pageID, FileOrigin& origin, Int64& position, Int32& walVersion)
  at LiteDB.Engine.DataService.<Read>d__5.MoveNext()
  at LiteDB.Engine.BufferReader..ctor(IEnumerable`1 source, Boolean utcDate)
  at LiteDB.Engine.DatafileLookup.Load(PageAddress rawId)
  at LiteDB.Engine.BasePipe.<LoadDocument>d__6.MoveNext()
  at LiteDB.Engine.QueryPipe.<Select>d__2.MoveNext()
  at LiteDB.Engine.QueryExecutor.<>c__DisplayClass10_0.<<ExecuteQuery>g__RunQuery|0>d.MoveNext()
  at LiteDB.BsonDataReader..ctor(IEnumerable`1 values, String collection)
  at LiteDB.Engine.QueryExecutor.ExecuteQuery(Boolean executionPlan)
  at LiteDB.Engine.LiteEngine.Query(String collection, Query query)
  at LiteDB.LiteQueryable`1.<ToDocuments>d__26.MoveNext()
  at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
  at Acars.SimObjects.AirportDatabase.Load(String folder, SimType type)
  at Acars.SimObjects.AirportDatabase.Load()
  at Acars.MainWindow.<metroWindow_ContentRendered>d__26.MoveNext()
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
  at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
  at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
  at System.Windows.Threading.DispatcherOperation.InvokeImpl()
  at MS.Internal.CulturePreservingExecutionContext.CallbackWrapper(Object obj)
  at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
  at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
  at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
  at MS.Internal.CulturePreservingExecutionContext.Run(CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
  at System.Windows.Threading.DispatcherOperation.Invoke()
  at System.Windows.Threading.Dispatcher.ProcessQueue()
  at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
  at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
  at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
  at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
  at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
  at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
  at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
  at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
  at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
  at System.Windows.Application.RunDispatcher(Object ignore)
  at System.Windows.Application.RunInternal(Window window)
  at Acars.App.Main()

Looks like there were 9+ crashes on this one. The associated code (extra stuff removed).

lock (_lock) {
    var dbPath = GetDbName(type);

    using (var db = new LiteDatabase(dbPath)) {
        var airports = db.GetCollection<FsAirport>("airports");
        var apts = airports.FindAll();

        foreach (var apt in apts) {
            Airports.Add(apt);
        }
    }
}

Then on saving:

Package: Acars
Version: 1.0.0.821
OS: WINDOWS
Windows: 10.0.18363
Date: 2020-02-21T01:17:29.4764536Z

Exception Stack:
LiteDB.LiteException: Invalid IndexPage buffer on 0
  at LiteDB.Engine.IndexPage..ctor(PageBuffer buffer)
  at LiteDB.Engine.BasePage.ReadPage[T](PageBuffer buffer)
  at LiteDB.Engine.Snapshot.ReadPage[T](UInt32 pageID, FileOrigin& origin, Int64& position, Int32& walVersion)
  at LiteDB.Engine.Snapshot.GetPage[T](UInt32 pageID, FileOrigin& origin, Int64& position, Int32& walVersion)
  at LiteDB.Engine.IndexService.AddNode(CollectionIndex index, BsonValue key, PageAddress dataBlock, Byte level, IndexNode last)
  at LiteDB.Engine.IndexService.AddNode(CollectionIndex index, BsonValue key, PageAddress dataBlock, IndexNode last)
  at LiteDB.Engine.LiteEngine.InsertDocument(Snapshot snapshot, BsonDocument doc, BsonAutoId autoId, IndexService indexer, DataService data)
  at LiteDB.Engine.LiteEngine.<>c__DisplayClass30_0.<Upsert>b__0(TransactionService transaction)
  at LiteDB.Engine.LiteEngine.AutoTransaction[T](Func`2 fn)
  at LiteDB.LiteCollection`1.Upsert(T entity)
  at Acars.SimObjects.AirportDatabase.Save()
  at Acars.ViewModels.SettingsViewModel.<ResyncScenery>b__37_0()
  at System.Threading.Tasks.Task.Execute()
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
  at Acars.ViewModels.SettingsViewModel.<ResyncScenery>d__37.MoveNext()
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
  at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
  at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
  at System.Windows.Threading.DispatcherOperation.InvokeImpl()
  at MS.Internal.CulturePreservingExecutionContext.CallbackWrapper(Object obj)
  at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
  at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
  at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
  at MS.Internal.CulturePreservingExecutionContext.Run(CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
  at System.Windows.Threading.DispatcherOperation.Invoke()
  at System.Windows.Threading.Dispatcher.ProcessQueue()
  at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
  at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
  at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
  at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
  at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
  at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
  at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
  at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
  at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
  at System.Windows.Application.RunDispatcher(Object ignore)
  at System.Windows.Application.RunInternal(Window window)
  at Acars.App.Main()

This save code is:

lock (_lock) {
    var dbFile = GetDbName(SimType);

    // Delete the old db file. Don't bother with updating rows
    if(File.Exists(dbFile)) {
        File.Delete(dbFile);
    }

    using (var db = new LiteDatabase(dbFile)) {
        var airports = db.GetCollection<FsAirport>("airports");
        var runways = db.GetCollection<FsRunway>("runways");
        var gates = db.GetCollection<FsGate>("gates");

        airports.EnsureIndex("ICAO");
        runways.EnsureIndex("ICAO");
        gates.EnsureIndex("ICAO");

        //airports.InsertBulk(Airports);
        foreach(var apt in Airports) {
            airports.Upsert(apt);

            if (apt.Runways.Count > 0) {
                runways.InsertBulk(apt.Runways);
            }

            if (apt.Gates.Count > 0) {
                gates.InsertBulk(apt.Gates);
            }
        }
    }
}

I do have this run code, but it's hard to tell if it's running from within this - it runs a Save() and then a Load() to switch to the newer database.


await Task.Run(() => {
    _airportDatabase.CreateDatabase(_appStateManager.Settings.SimType);

    // If they changed the SimType, reload the airport database
    if (_airportDatabase.SimType != _appStateManager.Settings.SimType) {
        _airportDatabase.Load();
    }
});

So I am deleting the old database file before saving. Maybe this isn't the best idea. But this code hasn't changed since using 4.x

Feb 21 '20 16:02 nabeelio

And after this error occurs, if you open in LiteDB.Studio and run a simple SELECT $ FROM collection you see if you lost any data?

Feb 21 '20 16:02 mbdavid

@mbdavid I'm trying to get my hands on the DB files, I will try that out

Feb 21 '20 16:02 nabeelio

It doesn't look like any data is lost. I have the db file and the -log.db file, if you want to see/examine.

Feb 21 '20 17:02 nabeelio

Humm... much better. Maybe was a clean buffer page in a wrong moment. I will review this and add more error detail message so, if occurs again, will be better to track where this buffer came from

Feb 21 '20 17:02 mbdavid

Ok great, thanks. I'm updating to 5.0.2 and going to see if that helps the affected client. Is this code-path unique to upsert?

Feb 21 '20 18:02 nabeelio

Sorry, I didn't understand this: "Is this code-path unique to upsert?" v5.0.2 is compatible with v5.0.1...

Feb 21 '20 18:02 mbdavid

Sorry - I mean these calls:

LiteDB.LiteException: Invalid IndexPage buffer on 0
  at LiteDB.Engine.IndexPage..ctor(PageBuffer buffer)

Is that only with the upsert? Or would it make the same calls with an Update and then Insert call? Just trying to think of ways to duplicate or if it will mitigate the problem.

Feb 21 '20 18:02 nabeelio

It's not about Upsert. Upsert is only a Update + Insert command. This occurs in adding a new index node (possible for _id). Engine try load a page from disk to memory and this page are complete empty (all 8192 zero bytes).

I'm releasing v5.0.3 with more debug message in this error, so you can upgrade direct to 5.0.3 so we can track better.

Feb 21 '20 18:02 mbdavid

Sounds good, thanks!

Feb 21 '20 18:02 nabeelio

Done. If you could update to v5.0.3 and paste stacktrace in next error will be great. Thanks

Feb 21 '20 19:02 mbdavid

Done, I will hopefully hear back soon.

Feb 21 '20 19:02 nabeelio

@mbdavid

In AppCenter I can see some those exceptions, which were added with this issue (InvalidPageType). I have similar stacktraces as @nabeelio has. And I also can't reproduce it, but I can see exceptions from other users in AppCenter.

Version: 5.0.5 TargetFramework: netstandard1.3, uap10240.

Exception 1 (On Read)

Message:

LiteDB.LiteException: Invalid Collection on 0. Full zero: True. Page Type: Empty. Prev/Next: 0/0. UniqueID: 1. ShareCounter: 1.

Stacktrace:

LiteDB.Engine.CollectionPage
LiteDB.Engine.CollectionPage..ctor(Engine.PageBuffer buffer)
MyApp!<BaseAddress>+0x16a2280
MyApp!<BaseAddress>+0x16a2174
MyApp!<BaseAddress>+0x16a2057
LiteDB.Engine
CollectionService.Get (String name, Boolean addIfNotExists, Engine.CollectionPage& collectionPage)
LiteDB.Engine.Snapshot
LiteDB.Engine.Snapshot..ctor(Engine.LockMode mode, String collectionName, Engine.HeaderPage header, UInt32 transactionID, Engine.TransactionPages transPages, Engine.LockService locker, Engine.WalIndexService walIndex, Engine.DiskReader reader, Boolean addIfNotExists)
LiteDB.Engine
TransactionService.<CreateSnapshot>g__create|42_0 (Engine.TransactionService.<>c__DisplayClass42_0& <unknown>)
LiteDB.Engine
TransactionService.CreateSnapshot (Engine.LockMode mode, String collection, Boolean addIfNotExists)
LiteDB.Engine.QueryExecutor.<>c__DisplayClass10_0
<<ExecuteQuery>g__RunQuery|0>d.MoveNext ()
LiteDB.BsonDataReader
LiteDB.BsonDataReader..ctor(Collections.Generic.IEnumerable<LiteDB.BsonValue> values, String collection)
LiteDB.Engine
QueryExecutor.ExecuteQuery (Boolean executionPlan)
LiteDB.Engine
QueryExecutor.ExecuteQuery ()
LiteDB.Engine
LiteEngine.Query (String collection, Query query)
LiteDB.LiteQueryable<LiteDB.LiteFileInfo<System
String>>.ExecuteReader ()
LiteDB.LiteQueryable<LiteDB.LiteFileInfo<System.String>>
<ToDocuments>d__26.MoveNext ()
System.Linq.Enumerable.WhereSelectEnumerableIterator$2<System.__Canon,System
__Canon>.MoveNext () f:\dd\ndp\fxcore\Open\src\System.Linq\src\System\Linq\Enumerable.cs at 359:25
System.Collections.Generic.List$1<System.__Canon>
f:\dd\ndp\fxcore\src\System.Collections\System\Collections\Generic\List.cs at 90:21
MyApp!<BaseAddress>+0x159c0ce
MyApp.Data.Repositories.BaseLiteDatabaseRepository<System.String,MyApp.Shared.Repositories
TorrServerEntity>.GetAll ()
MyApp!<BaseAddress>+0x1bfd6f2

Exception 2 (On Upsert)

Message:

LiteDB.LiteException: Invalid Collection on 0. Full zero: True. Page Type: Empty. Prev/Next: 0/0. UniqueID: 2. ShareCounter: -1.

Stacktrace:

LiteDB.Engine.CollectionPage
LiteDB.Engine.CollectionPage..ctor(Engine.PageBuffer buffer)
MyApp!<BaseAddress>+0x16a2280
MyApp!<BaseAddress>+0x16a2174
MyApp!<BaseAddress>+0x16a2057
LiteDB.Engine
CollectionService.Get (String name, Boolean addIfNotExists, Engine.CollectionPage& collectionPage)
LiteDB.Engine.Snapshot
LiteDB.Engine.Snapshot..ctor(Engine.LockMode mode, String collectionName, Engine.HeaderPage header, UInt32 transactionID, Engine.TransactionPages transPages, Engine.LockService locker, Engine.WalIndexService walIndex, Engine.DiskReader reader, Boolean addIfNotExists)
LiteDB.Engine
TransactionService.<CreateSnapshot>g__create|42_0 (Engine.TransactionService.<>c__DisplayClass42_0& <unknown>)
LiteDB.Engine
TransactionService.CreateSnapshot (Engine.LockMode mode, String collection, Boolean addIfNotExists)
LiteDB.Engine.QueryExecutor.<>c__DisplayClass10_0
<<ExecuteQuery>g__RunQuery|0>d.MoveNext ()
LiteDB.BsonDataReader
LiteDB.BsonDataReader..ctor(Collections.Generic.IEnumerable<LiteDB.BsonValue> values, String collection)
LiteDB.Engine
QueryExecutor.ExecuteQuery (Boolean executionPlan)
LiteDB.Engine
QueryExecutor.ExecuteQuery ()
LiteDB.Engine
LiteEngine.Query (String collection, Query query)
LiteDB.LiteQueryable<LiteDB.LiteFileInfo<System
String>>.ExecuteReader ()
LiteDB.LiteQueryable<LiteDB.LiteFileInfo<System.String>>
<ToDocuments>d__26.MoveNext ()
System.Linq.Enumerable.WhereSelectEnumerableIterator$2<System.__Canon,System
__Canon>.MoveNext () f:\dd\ndp\fxcore\Open\src\System.Linq\src\System\Linq\Enumerable.cs at 359:25
System.Collections.Generic.List$1<System.__Canon>
f:\dd\ndp\fxcore\src\System.Collections\System\Collections\Generic\List.cs at 90:21
MyApp!<BaseAddress>+0x159c0ce
MyApp.Data.Repositories.BaseLiteDatabaseRepository<System.String,MyApp.Shared.Repositories
TorrServerEntity>.GetAll ()
MyApp!<BaseAddress>+0x1bfd6f2

Repository pattern methods, used in stacktraces above:

public virtual IAsyncEnumerable<TEntity> GetAll()
{
    var result = Collection.FindAll().ToList();
    return result.ToAsyncEnumerable();
}
public virtual ValueTask UpsertManyAsync(IEnumerable<TEntity> items)
{
    Collection.Upsert(items);
    return new ValueTask();
}

Note: both are 'async' by interface contract, but in LiteDB implementation both are sync.

Model:

public class TorrServerEntity
{
    public string? TorrentId { get; set; }

    public string? TorrServerHash { get; set; }
}

Database.Mapper.Entity<TorrServerEntity>()
    .Id(i => i.TorrServerHash, false);

I hope, it will help in some way. Tell me, if you need additional information.

Apr 09 '20 06:04 maxkatz6

I haven't gotten any crash reports since the 12th, I'm glad you're seeing something.

Apr 09 '20 20:04 nabeelio

Oh, I just saw your PR for the missing throw, perhaps that's why. Also getting crash reports in AppCenter

Apr 09 '20 20:04 nabeelio

@mbdavid @lbnascimento hi! Maybe you know something about that?

Unfortunately, I can't test it with 5.0.7 in near time.

Apr 14 '20 23:04 maxkatz6

Same error with 5.0.7:

LiteDB.LiteException: Invalid Collection on 0. Full zero: True. Page Type: Empty. Prev/Next: 0/0. UniqueID: 4. ShareCounter: 1.

LiteDB.Engine.CollectionPage
at 0
LiteDB.Engine.BasePage
ReadPage<System.__Canon> (Void* InstParam, $PageBuffer buffer) at 0
LiteDB.Engine.Snapshot
ReadPage<System.__Canon> (Void* InstParam, UInt32 pageID, $FileOrigin& origin, Int64& position, Int32& walVersion) at 0
LiteDB.Engine.Snapshot
GetPage<System.__Canon> (Void* InstParam, UInt32 pageID, $FileOrigin& origin, Int64& position, Int32& walVersion) at 0
LiteDB.Engine
CollectionService.Get (String name, Boolean addIfNotExists, $CollectionPage& collectionPage) at 0
LiteDB.Engine.Snapshot
at 0
LiteDB.Engine
TransactionService.<CreateSnapshot>g__create|42_0 ($<>c__DisplayClass42_0 Param1) at 0
LiteDB.Engine
TransactionService.CreateSnapshot ($LockMode mode, String collection, Boolean addIfNotExists) at 0
LiteDB.Engine.QueryExecutor.<>c__DisplayClass10_0
<<ExecuteQuery>g__RunQuery|0>d.MoveNext () at 0
LiteDB.BsonDataReader
at 0
LiteDB.Engine
QueryExecutor.ExecuteQuery (Boolean executionPlan) at 0
LiteDB.Engine
QueryExecutor.ExecuteQuery () at 0
LiteDB.Engine
LiteEngine.Query (String collection, $Query query) at 0
LiteDB.LiteQueryable$1<System
__Canon>.ExecuteReader () at 0
LiteDB.LiteQueryable$1<System.__Canon>
<ToDocuments>d__26.MoveNext () at 0
System.Linq.Enumerable.WhereSelectEnumerableIterator$2<System.__Canon,System
__Canon>.MoveNext () f:\dd\ndp\fxcore\Open\src\System.Linq\src\System\Linq\Enumerable.cs at 359:25
System.Collections.Generic.List$1<System.__Canon>
f:\dd\ndp\fxcore\src\System.Collections\System\Collections\Generic\List.cs at 90:21
System.Linq.Enumerable
ToList<System.__Canon> (Void* InstParam, IEnumerable$1<__Canon> source) at 0
BaseLiteDatabaseRepository$2<System.Guid,System
__Canon>.GetAll ()

Apr 22 '20 23:04 maxkatz6

I'm seeing this:

2020-04-26 21:14:28.5769|ERROR|Acars.App|UnhandledException caught : Invalid Index on 0. Full zero: True. Page Type: Empty. Prev/Next: 0/0. UniqueID: 8. ShareCounter: -1. 
2020-04-26 21:14:28.6918|ERROR|Acars.App|UnhandledException StackTrace :    at LiteDB.Engine.IndexPage..ctor(PageBuffer buffer)
   at LiteDB.Engine.BasePage.ReadPage[T](PageBuffer buffer)
   at LiteDB.Engine.Snapshot.ReadPage[T](UInt32 pageID, FileOrigin& origin, Int64& position, Int32& walVersion)
   at LiteDB.Engine.Snapshot.GetPage[T](UInt32 pageID, FileOrigin& origin, Int64& position, Int32& walVersion)
   at LiteDB.Engine.IndexService.AddNode(CollectionIndex index, BsonValue key, PageAddress dataBlock, Byte level, IndexNode last)
   at LiteDB.Engine.IndexService.AddNode(CollectionIndex index, BsonValue key, PageAddress dataBlock, IndexNode last)
   at LiteDB.Engine.LiteEngine.InsertDocument(Snapshot snapshot, BsonDocument doc, BsonAutoId autoId, IndexService indexer, DataService data)
   at LiteDB.Engine.LiteEngine.<>c__DisplayClass30_0.<Upsert>b__0(TransactionService transaction)
   at LiteDB.Engine.LiteEngine.AutoTransaction[T](Func`2 fn)
   at LiteDB.LiteCollection`1.Upsert(T entity)
   at Acars.SimObjects.AirportDatabase.Save()
   at Acars.SimObjects.AirportDatabase.CreateDatabase(SimType type)
   at Acars.ViewModels.SettingsViewModel.<SyncScenery>b__37_0()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Acars.ViewModels.SettingsViewModel.<SyncScenery>d__37.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
   at System.Windows.Threading.DispatcherOperation.InvokeImpl()
   at MS.Internal.CulturePreservingExecutionContext.CallbackWrapper(Object obj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at MS.Internal.CulturePreservingExecutionContext.Run(CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Windows.Threading.DispatcherOperation.Invoke()
   at System.Windows.Threading.Dispatcher.ProcessQueue()
   at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
   at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
   at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
   at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
   at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
   at System.Windows.Application.RunDispatcher(Object ignore)
   at System.Windows.Application.RunInternal(Window window)
   at Acars.App.Main()

Apr 26 '20 19:04 nabeelio

I'm getting it on this file, trying to read the settings table. Loading it in the GUI shows the error.

settings.zip

May 01 '20 20:05 nabeelio

seeing the same issue: "Invalid Index on 0. Full zero: True. Page Type: Empty. Prev/Next: 0/0. UniqueID: 2. ShareCounter: 10. " Any ideas?

Jun 17 '20 07:06 nikun5

Problem still exists in v5.0.9. Same error on Litedb Studio.

Aug 06 '20 22:08 charly006

And after this error occurs, if you open in LiteDB.Studio and run a simple SELECT $ FROM collection you see if you lost any data?

@mbdavid Invalid Data on 0. Full zero: True. Page Type: Empty. Prev/Next: 0/0. UniqueID: 3. ShareCounter: 1. No data is displayed anymore on Litedb Studio

Aug 07 '20 06:08 charly006

Am curious if there is any update on this issue, I've just started to run into the same problem and may have to backout and may need to change database solutions if this is a persistent error

This was opened in February and still appears to be a problem any ideas about a timeframe for resolution

From what I've read i think this may only be an issue when the upgrade flag is set - I'll do some further investigation and report if this is a problem when i switch it off

Aug 10 '20 03:08 pomeara

From what I've read i think this may only be an issue when the upgrade flag is set

No. I don't have this flag set in my application, but exception still occurs for my users.

Aug 10 '20 05:08 maxkatz6

@mbdavid Any updates? Going to have to rip this out and switch to something else if there's no resolution soon.

Summary
LiteDB.LiteException: Invalid Collection on 0. Full zero: True. Page Type: Empty. Prev/Next: 0/0. UniqueID: 1. ShareCounter: 1.

Aug 21 '20 22:08 nabeelio

@mbdavid: Same problem here. It must be a race condition between disposing of a LiteDatabase object and creating a new one on a new request. Is there a way to force LiteDB to flush the WAL manually after save to prove this?

Aug 25 '20 21:08 t00

I found my problem when away after I fixed the keys I was using for the database - i.e. dont use a string index as soon as I corrected the index this was no longer a problem

Aug 25 '20 21:08 pomeara

I managed to fix it by switching service containing LiteDatabase to a singleton so that only one instance of LiteDatabase is used for all requests. Obviously this is not perfect as a log file will be filled in with changes instead of a main db file but a delayed dispose + create with a SemaphoreSlim lock should do the trick of merging log into the main db. Assuming LiteDb log is consistent, there is no risk of data loss with this temporary "solution".

Aug 25 '20 22:08 t00

Hello, I have similar issue. After few hours I'm getting the following:

LiteDB.LiteException: Invalid Data on 758. Full zero: False. Page Type: Empty. Prev/Next: 4294967295/743. UniqueID: 314063. ShareCounter: -1. at LiteDB.Engine.DataPage..ctor(PageBuffer buffer) at LiteDB.Engine.BasePage.ReadPage[T](PageBuffer buffer) at LiteDB.Engine.Snapshot.ReadPage[T](UInt32 pageID, FileOrigin& origin, Int64& position, Int32& walVersion) at LiteDB.Engine.Snapshot.GetPage[T](UInt32 pageID, FileOrigin& origin, Int64& position, Int32& walVersion) at LiteDB.Engine.Snapshot.GetFreeDataPage(Int32 bytesLength) at LiteDB.Engine.DataService.<>c__DisplayClass3_0.<<Insert>g__source|0>d.MoveNext() at LiteDB.Engine.BufferWriter..ctor(IEnumerable'1 source) at LiteDB.Engine.DataService.Insert(BsonDocument doc) at LiteDB.Engine.LiteEngine.InsertDocument(Snapshot snapshot, BsonDocument doc, BsonAutoId autoId, IndexService indexer, DataService data) at LiteDB.Engine.LiteEngine.<>c__DisplayClass7_0.<Insert>b__0(TransactionService transaction) at LiteDB.Engine.LiteEngine.AutoTransaction[T](Func'2 fn) at LiteDB.LiteCollection'1.Insert(T entity)

the application manages time series: it has several parallel threads that are inserting records into the same database instance. Every thread is working on its collection. Each thread, every 10s, removes old records in order to keep a history of 1 hour.

it seems there is no solution yet, right?

Sep 02 '20 06:09 stefanocadoaleph

I got a similar error message when doing multithreaded tests to see how thread safe LiteDB is

An unhandled exception of type 'LiteDB.LiteException' occurred in LiteDB.dll Additional information: Invalid Collection on 0. Full zero: True. Page Type: Empty. Prev/Next: 0/0. UniqueID: 5. ShareCounter: 1.

I got this error when trying to read from a collection while writing to the same collection from a different thread. Sample code that I was using is copied below.

using LiteDB;
using System;
using System.Linq;
using System.Threading;

namespace ConsoleLiteDbTest
{
    class Program
    {
        static void Main(string[] args)
        {
            using (var database = new LiteDatabase(@"Filename=c:\test\multithreadedTests.db;"))
            {
                var readThread = new Thread(new ThreadStart(() => { readContinuously(database); }));
                var writeThread = new Thread(new ThreadStart(() => { writeContinuously(database); }));
                readThread.Start();
                writeThread.Start();
                writeThread.Join();
                readThread.Join();

                Console.WriteLine("Press any key to continue");
                Console.ReadKey();
            }
        }

        static void readContinuously(LiteDatabase database)
        {
            int recordsRead = -1;
            while (recordsRead < 100)
            {
                var collection = database.GetCollection<TestObject>("Testing");
                var records = collection.FindAll();
                var recordCount = records.Count(); //break point occurred here with the error message above.
                if (recordCount != recordsRead)
                {
                    recordsRead = recordCount;
                    Console.WriteLine($"Read in {recordsRead} records.");
                }
            }
        }

        static void writeContinuously(LiteDatabase database)
        {
            for (var i = 0; i < 100; i++)
            {
                Thread.Sleep(10);
                var collection = database.GetCollection<TestObject>("Testing");
                Console.WriteLine($"Inserting object \"{i}\"");
                collection.Insert(new TestObject { Value = $"This is object {i}" });
            }
        }

        class TestObject
        {
            public string Value { get; set; }
            public override string ToString()
            {
                return Value;
            }
        }
    }
}

Sep 04 '20 21:09 trockefeller-pathway

Same issue for me

Sep 16 '20 07:09 Codelisk

LiteDB LiteDB copied to clipboard

[BUG] Intermittent Invalid IndexPage/DataPage buffer on 0

Exception 1 (On Read)

Message:

Stacktrace:

Exception 2 (On Upsert)

Message:

Stacktrace:

LiteDB
LiteDB copied to clipboard