isar
isar copied to clipboard
isar
IsarError: MdbxError (-30779): MDBX_PROBLEM: Unexpected internal error, transaction should be aborted
Steps to Reproduce
We couldn't really reproduce this error at the moment. My client ran into this problem when his computer ran out of power and shut dow.
Code sample
we don't have a way to reproduce it.
Details
- Platform: MacOS 12.7.6.
- Flutter version: 3.24.5.
- Isar version: 3.1.0+1.
- [x] I searched for similar issues already
- [x] I filled the details section with the exact device model and version
- [ ] I am able to provide a reproducible example
Probably he was in the middle of a writeTxn-(the database is in a locked state) when his computer shutdown. And I believe you cannot start another transaction on a locked Isar db.
I faced a similar issue, and unfortunately, there's no sure fix for this, but a workaround is to simply delete the locked Isar database and recreate it.
For example, this is how I open my isar instance:
Isar? _globalIsar;
String? _currentInstanceName;
class IsarLocalStorageImpl<T extends IsarModel> {
IsarLocalStorageImpl._({String? name, Directory? directory})
: _name = name ?? defaultName,
_directory = directory ?? defaultDirectory;
static final defaultDirectory = Directory(p.join(defaultDirectoryRoot, 'db', 'isar', 'main'));
static final defaultDirectoryRoot = Static.cacheDir!.path;
static const int defaultMaxSizeMiB = 2048;
static const defaultName = 'my_app';
final Directory _directory;
final String _name;
@override
String toString() => 'IsarLocalStorageImpl(name: $_name, directory: $_directory, isGlobalReady: $isGlobalReady)';
Isar get _isar {
try {
return _globalIsar!;
} catch (e, st) {
log.s('Tried to access a NULL Object [_globalIsarInstance]', error: e, stackTrace: st, toConsole: true);
try {
final instance = Isar.getInstance(_name);
return instance!;
} catch (e) {
log.s('Isar instance not found: $_name', error: e, stackTrace: st, toConsole: true);
rethrow;
}
}
}
/// Used to check if the Global isar instance is ready to use
static bool get isGlobalReady => _globalIsar != null;
// @protected
// @visibleForTesting
Isar get isar => _isar;
/// Returns the existing global instance of [IsarLocalStorageImpl] with the specified type [T].
///
/// @throws [AssertionError] if the instance is null.
static IsarLocalStorageImpl<T> instance<T extends IsarModel>() {
assert(
_globalIsar != null,
'Isar has not been initialized from [IsarLocalStorageImpl].\n'
'Consider calling [IsarLocalStorageImpl.init(global: true)] first.',
);
return IsarLocalStorageImpl<T>._(name: _globalIsar!.name, directory: Directory(_globalIsar!.directory!));
}
/// Opens an Isar instance with the specified [directory] and [name].
///
/// - `path`: The path, (usually) relative to `IsarLocalStorageImpl.defaultDirectoryRoot` where the database file should be stored. By default, `IsarLocalStorageImpl.defaultDirectory.path` is used.
/// - `name`: Open multiple instances with distinct names. By default, `IsarLocalStorageImpl.defaultName` is used.
/// - `schemas`: is a list of [CollectionSchema] that defines the schema of the collections in the Isar instance.
/// - `relaxedDurability`: Relaxes the durability guarantee to increase write performance. In case of a system crash (not app crash), it is possible to lose the last committed transaction. Corruption is not possible
/// - `inspector`: Enables the Isar Inspector. The inspector is a tool to inspect the content of the database and run queries.
/// - `maxSizeMiB`: The maximum size of the database in MiB. The default is 2048 MiB.
/// - `compactOnLaunch`: Isar databases can contain unused space that will be reused for later operations. You can specify conditions to trigger manual compaction where the entire database is copied and unused space freed.
/// This operation can only be performed while a database is being opened and should only be used if absolutely necessary.
/// - `isAbsolutePath`: If `true`, [path] is treated as an absolute path. Default is `false`.
/// - `global`: If `true`, the created instance is stored as a global Isar instance and can be accessed using `IsarLocalStorageImpl.instance()`.
static FutureOr<IsarLocalStorageImpl<T>> init<T extends IsarModel>({
String? path,
List<CollectionSchema<Object?>> schemas = const [],
String? name,
bool? relaxedDurability,
bool? inspector,
int? maxSizeMiB,
CompactCondition? compactOnLaunch,
bool? isAbsolutePath,
bool? global,
bool? disposeBeforeInit,
}) async {
name ??= defaultName;
isAbsolutePath ??= (path != null && path.startsWith('/'));
global ??= false;
final directory = () {
if (path != null) {
if (isAbsolutePath!) {
return Directory(path);
} else {
return Directory(p.join(defaultDirectoryRoot, path));
}
}
return defaultDirectory;
}();
final isar = await _open(
directory,
schemas: schemas,
name: name,
inspector: inspector,
maxSizeMiB: maxSizeMiB,
compactOnLaunch: compactOnLaunch,
relaxedDurability: relaxedDurability,
disposeBeforeInit: disposeBeforeInit,
);
if (global) _globalIsar = isar;
return IsarLocalStorageImpl._(name: isar.name, directory: Directory(isar.directory!));
}
static Future<void> close(String name, {bool deleteFromDisk = false}) async {
if (!Isar.instanceNames.contains(name)) return;
final instance = Isar.getInstance(name);
await instance?.close(deleteFromDisk: deleteFromDisk);
if (deleteFromDisk) await instance?.directory?.let(Directory.new).delete(recursive: true);
}
Future<bool> dispose({bool deleteFromDisk = false}) async {
try {
await _isar.close(deleteFromDisk: deleteFromDisk);
if (deleteFromDisk) await _directory.delete(recursive: true);
return true;
} catch (_) {
return false;
}
}
static Future<Isar> _open(
Directory directory, {
List<CollectionSchema<Object?>> schemas = const [],
String? name,
bool? relaxedDurability,
bool? inspector,
int? maxSizeMiB,
CompactCondition? compactOnLaunch,
bool? createDirectory,
bool? disposeBeforeInit,
}) async {
name ??= defaultName;
inspector ??= true;
createDirectory ??= true;
relaxedDurability ??= true;
disposeBeforeInit ??= true;
maxSizeMiB ??= Isar.defaultMaxSizeMiB;
var retryCount = 0;
const maxRetry = 5;
if (name.contains(defaultName)) name = await _getCachedIsarInstanceName(name);
if (createDirectory) await _createDirectory(directory);
final collectionSchemas = [
...globalIsarSchemas,
//..add more schemas
...schemas.filter((s) => !globalIsarSchemas.contains(s)),
];
if (disposeBeforeInit) await IsarLocalStorageImpl.close(name);
late Isar? _isar;
final oldName = name;
await Future.doWhile(() async {
try {
if (retryCount >= maxRetry) {
// If the maximum number of retries has been reached,
// we can assume the Isar instance could not be opened probably due to "https://github.com/isar/isar/issues/570"
// the only other option is to delete the directory & change the name of the Isar instance.
if (name!.contains(defaultName)) {
await _createDirectory(directory, deleteIfExists: true);
name = await _getCachedIsarInstanceName(name!);
}
}
if (oldName == name) {
debugPrint('Reusing existing Isar instance [$oldName]');
} else {
debugPrint('Opening a new Isar instance [$name] - count: $retryCount\n'
'BuildEnvironment: ${env.type}\n'
'BuildFlavor: ${env.flavor}');
}
/// If an open instance could not be found, then open a new instance.
_isar = await Isar.open(
collectionSchemas,
directory: directory.path,
name: name!,
relaxedDurability: relaxedDurability!,
compactOnLaunch: compactOnLaunch,
inspector: inspector!,
maxSizeMiB: maxSizeMiB!,
);
return false;
} catch (ex, st) {
debugPrint('[DEBUG] - Failed to open Isar instance [$name]..But retrying after 500ms: $ex\n$st');
await Future.delayed(const Duration(milliseconds: 500));
retryCount++;
return retryCount <= maxRetry;
}
});
if (_isar == null) {
throw AssertionError('Failed to open Isar instance [$name] after $retryCount retries');
}
return _isar!;
}
static Future<String> _getCachedIsarInstanceName(String _default) async {
final currentName = await sharedPrefs?.read('current-isar-instance');
if (currentName != null) {
_default = currentName;
} else {
_default = '$_default-${localNow.millisecondsSinceEpoch}';
await sharedPrefs?.write('current-isar-instance', _default);
}
_currentInstanceName = _default;
return _default;
}
static Future<void> _createDirectory(Directory directory, {bool? deleteIfExists}) async {
deleteIfExists ??= false;
final dirExists = await directory.exists();
if (!dirExists) {
await directory.create(recursive: true);
} else if (dirExists && deleteIfExists) {
await directory.delete(recursive: true);
await directory.create(recursive: true);
}
}
}
The trick is in the method => _open()
I hope this helps :)
Probably he was in the middle of a
writeTxn-(the database is in a locked state) when his computer shutdown. And I believe you cannot start another transaction on a locked Isar db.I faced a similar issue, and unfortunately, there's no sure fix for this, but a workaround is to simply delete the locked Isar database and recreate it.
For example, this is how I open my isar instance:
Isar? _globalIsar; String? _currentInstanceName; class IsarLocalStorageImpl<T extends IsarModel> { IsarLocalStorageImpl._({String? name, Directory? directory}) : _name = name ?? defaultName, _directory = directory ?? defaultDirectory; static final defaultDirectory = Directory(p.join(defaultDirectoryRoot, 'db', 'isar', 'main')); static final defaultDirectoryRoot = Static.cacheDir!.path; static const int defaultMaxSizeMiB = 2048; static const defaultName = 'my_app'; final Directory _directory; final String _name; @override String toString() => 'IsarLocalStorageImpl(name: $_name, directory: $_directory, isGlobalReady: $isGlobalReady)'; Isar get _isar { try { return _globalIsar!; } catch (e, st) { log.s('Tried to access a NULL Object [_globalIsarInstance]', error: e, stackTrace: st, toConsole: true); try { final instance = Isar.getInstance(_name); return instance!; } catch (e) { log.s('Isar instance not found: $_name', error: e, stackTrace: st, toConsole: true); rethrow; } } } /// Used to check if the Global isar instance is ready to use static bool get isGlobalReady => _globalIsar != null; // @protected // @visibleForTesting Isar get isar => _isar; /// Returns the existing global instance of [IsarLocalStorageImpl] with the specified type [T]. /// /// @throws [AssertionError] if the instance is null. static IsarLocalStorageImpl<T> instance<T extends IsarModel>() { assert( _globalIsar != null, 'Isar has not been initialized from [IsarLocalStorageImpl].\n' 'Consider calling [IsarLocalStorageImpl.init(global: true)] first.', ); return IsarLocalStorageImpl<T>._(name: _globalIsar!.name, directory: Directory(_globalIsar!.directory!)); } /// Opens an Isar instance with the specified [directory] and [name]. /// /// - `path`: The path, (usually) relative to `IsarLocalStorageImpl.defaultDirectoryRoot` where the database file should be stored. By default, `IsarLocalStorageImpl.defaultDirectory.path` is used. /// - `name`: Open multiple instances with distinct names. By default, `IsarLocalStorageImpl.defaultName` is used. /// - `schemas`: is a list of [CollectionSchema] that defines the schema of the collections in the Isar instance. /// - `relaxedDurability`: Relaxes the durability guarantee to increase write performance. In case of a system crash (not app crash), it is possible to lose the last committed transaction. Corruption is not possible /// - `inspector`: Enables the Isar Inspector. The inspector is a tool to inspect the content of the database and run queries. /// - `maxSizeMiB`: The maximum size of the database in MiB. The default is 2048 MiB. /// - `compactOnLaunch`: Isar databases can contain unused space that will be reused for later operations. You can specify conditions to trigger manual compaction where the entire database is copied and unused space freed. /// This operation can only be performed while a database is being opened and should only be used if absolutely necessary. /// - `isAbsolutePath`: If `true`, [path] is treated as an absolute path. Default is `false`. /// - `global`: If `true`, the created instance is stored as a global Isar instance and can be accessed using `IsarLocalStorageImpl.instance()`. static FutureOr<IsarLocalStorageImpl<T>> init<T extends IsarModel>({ String? path, List<CollectionSchema<Object?>> schemas = const [], String? name, bool? relaxedDurability, bool? inspector, int? maxSizeMiB, CompactCondition? compactOnLaunch, bool? isAbsolutePath, bool? global, bool? disposeBeforeInit, }) async { name ??= defaultName; isAbsolutePath ??= (path != null && path.startsWith('/')); global ??= false; final directory = () { if (path != null) { if (isAbsolutePath!) { return Directory(path); } else { return Directory(p.join(defaultDirectoryRoot, path)); } } return defaultDirectory; }(); final isar = await _open( directory, schemas: schemas, name: name, inspector: inspector, maxSizeMiB: maxSizeMiB, compactOnLaunch: compactOnLaunch, relaxedDurability: relaxedDurability, disposeBeforeInit: disposeBeforeInit, ); if (global) _globalIsar = isar; return IsarLocalStorageImpl._(name: isar.name, directory: Directory(isar.directory!)); } static Future<void> close(String name, {bool deleteFromDisk = false}) async { if (!Isar.instanceNames.contains(name)) return; final instance = Isar.getInstance(name); await instance?.close(deleteFromDisk: deleteFromDisk); if (deleteFromDisk) await instance?.directory?.let(Directory.new).delete(recursive: true); } Future<bool> dispose({bool deleteFromDisk = false}) async { try { await _isar.close(deleteFromDisk: deleteFromDisk); if (deleteFromDisk) await _directory.delete(recursive: true); return true; } catch (_) { return false; } } static Future<Isar> _open( Directory directory, { List<CollectionSchema<Object?>> schemas = const [], String? name, bool? relaxedDurability, bool? inspector, int? maxSizeMiB, CompactCondition? compactOnLaunch, bool? createDirectory, bool? disposeBeforeInit, }) async { name ??= defaultName; inspector ??= true; createDirectory ??= true; relaxedDurability ??= true; disposeBeforeInit ??= true; maxSizeMiB ??= Isar.defaultMaxSizeMiB; var retryCount = 0; const maxRetry = 5; if (name.contains(defaultName)) name = await _getCachedIsarInstanceName(name); if (createDirectory) await _createDirectory(directory); final collectionSchemas = [ ...globalIsarSchemas, //..add more schemas ...schemas.filter((s) => !globalIsarSchemas.contains(s)), ]; if (disposeBeforeInit) await IsarLocalStorageImpl.close(name); late Isar? _isar; final oldName = name; await Future.doWhile(() async { try { if (retryCount >= maxRetry) { // If the maximum number of retries has been reached, // we can assume the Isar instance could not be opened probably due to "https://github.com/isar/isar/issues/570" // the only other option is to delete the directory & change the name of the Isar instance. if (name!.contains(defaultName)) { await _createDirectory(directory, deleteIfExists: true); name = await _getCachedIsarInstanceName(name!); } } if (oldName == name) { debugPrint('Reusing existing Isar instance [$oldName]'); } else { debugPrint('Opening a new Isar instance [$name] - count: $retryCount\n' 'BuildEnvironment: ${env.type}\n' 'BuildFlavor: ${env.flavor}'); } /// If an open instance could not be found, then open a new instance. _isar = await Isar.open( collectionSchemas, directory: directory.path, name: name!, relaxedDurability: relaxedDurability!, compactOnLaunch: compactOnLaunch, inspector: inspector!, maxSizeMiB: maxSizeMiB!, ); return false; } catch (ex, st) { debugPrint('[DEBUG] - Failed to open Isar instance [$name]..But retrying after 500ms: $ex\n$st'); await Future.delayed(const Duration(milliseconds: 500)); retryCount++; return retryCount <= maxRetry; } }); if (_isar == null) { throw AssertionError('Failed to open Isar instance [$name] after $retryCount retries'); } return _isar!; } static Future<String> _getCachedIsarInstanceName(String _default) async { final currentName = await sharedPrefs?.read('current-isar-instance'); if (currentName != null) { _default = currentName; } else { _default = '$_default-${localNow.millisecondsSinceEpoch}'; await sharedPrefs?.write('current-isar-instance', _default); } _currentInstanceName = _default; return _default; } static Future<void> _createDirectory(Directory directory, {bool? deleteIfExists}) async { deleteIfExists ??= false; final dirExists = await directory.exists(); if (!dirExists) { await directory.create(recursive: true); } else if (dirExists && deleteIfExists) { await directory.delete(recursive: true); await directory.create(recursive: true); } } }The trick is in the method =>
_open()I hope this helps :)
Thank you.
The MDBX_PROBLEM error is returned from libmdbx (the key-value storage engine written in C).
As the main developer of libmdbx, I want to give a few explanations:
- libmdbx don't use any techniques which could lead to "leave database in a locked state" after the unexpected shutdown, system/power failure, etc. Actually libmdbx reliably purge any "locks" (re-create the lck file) when opening the database, unless it is already open by another process (this is one of many improvements beyond LMDB).
- However, if the DB is intentionally opened in an unsafe/fragile mode (see documentation), then in the event of a system failure or unexpected power outage, especially on consumer-grade NVMe/SSD/HDD (without capacitors or backup power), the database may be corrupted (i.e. when not all data was saved on the media) and
MDBX_PROBLEMcan be returned in subsequent operations. In such cases, the database should be checked bymdbx_chktool (the source code and man-pages are bundled within the libmdbx's source code), and in some cases, recovery is possible (switching to the previous MVCC-snapshot if it remains intact). - libmdbx provides logging of additional information in case of such errors, but apparently this logging is not involved in Isar (or used bindings), so the causes of the error are not known.
- Nonetheless, please use the latest libmdbx release on the origin above, either on github-mirror. Ask the maintainers and/or developers of Isar to update libmdbx used and/or contact the maintainers/developers of the bindings used to update the used library, etc.
- On the project's website mentioned above, there is a link to the telegram group, where you can quickly get advice.
