EXC_BAD_ACCESS fatal crash on iOS 17 when accessing store from within isolate

[JaffaKetchup]

trafficstars

Is there an existing issue?

• [x] I have searched existing issues
• Appears to be similar to #379, but possibly unrelated? See below.

Build info

• objectbox version: 4.0
• Flutter/Dart version: Flutter 3.22
• Build OS: Uncertain
• Deployment OS or device: Reported on iOS 17

Steps to reproduce

This report is based on a report to a library I own: https://github.com/JaffaKetchup/flutter_map_tile_caching/issues/159. Therefore, some information is limited as a 3rd party suffered the issue. I am providing as much information as possible, please let me know if more is required.

1. The issue appears to occur on iOS devices - unfortunately I do not have an iOS device to test
2. The last breakpoint hit appears to suggest the following code triggers the issue:

Code A

Future<int> deleteTiles({
    required Query<ObjectBoxStore> storesQuery,
    required Query<ObjectBoxTile> tilesQuery,
  }) async {
    final stores = root.box<ObjectBoxStore>();
    final tiles = root.box<ObjectBoxTile>();

    bool hadTilesToUpdate = false;
    int rootDeltaSize = 0;
    final tilesToRemove = <int>[];
    final storesToUpdate = <String, ObjectBoxStore>{};

    final queriedStores = storesQuery.property(ObjectBoxStore_.name).find();
    if (queriedStores.isEmpty) return 0;

    // For each store, remove it from the tile if requested
    // For each store & if removed, update that store's stats
    await for (final tile in tilesQuery.stream()) { // APPEARS TO CRASH HERE
      tile.stores.removeWhere((store) {
        if (!queriedStores.contains(store.name)) return false;

        storesToUpdate[store.name] = (storesToUpdate[store.name] ?? store)
          ..length -= 1
          ..size -= tile.bytes.lengthInBytes;

        return true;
      });

      if (tile.stores.isNotEmpty) {
        tile.stores.applyToDb(mode: PutMode.update);
        hadTilesToUpdate = true;
        continue;
      }

      rootDeltaSize -= tile.bytes.lengthInBytes;
      tilesToRemove.add(tile.id);
    }

    if (!hadTilesToUpdate && tilesToRemove.isEmpty) return 0;

    root.runInTransaction(
      TxMode.write,
      () {
        tilesToRemove.forEach(tiles.remove);

        updateRootStatistics(
          deltaLength: -tilesToRemove.length,
          deltaSize: rootDeltaSize,
        );

        stores.putMany(
          storesToUpdate.values.toList(),
          mode: PutMode.update,
        );
      },
    );

    return tilesToRemove.length;
  }

This code is inside an isolate, and is called also from within the same isolate, with the following arguments:

Code B

// This code is held within a switch statement executed by listening to a `RecievePort`

final String storeName = '<argument>';

final tiles = root.box<ObjectBoxTile>();
final stores = root.box<ObjectBoxStore>();

final storeQuery = stores.query(ObjectBoxStore_.name.equals(storeName)).build();
final store = storeQuery.findUnique() ?? (throw StoreNotExists(storeName: storeName));

final tilesQuery = (tiles.query()
        ..linkMany(
          ObjectBoxTile_.stores,
          ObjectBoxStore_.name.equals(storeName),
        ))
      .build();

deleteTiles(storesQuery: storeQuery, tilesQuery: tilesQuery).then((_) {
	// likely irrelevant, but effectively signals to the 'parent' 'main thread' isolate that the operation is complete
});

This looks similar to #379 (which was also related to usage in isolates), but it may not be. To open the stores in the new isolate, we use the new attach API, not fromReference. Maybe the issue is streaming in an isolate?

If more code is needed, all code is available on the FMTC repo. I can help filter out whatever is needed.

FMTC also uses the term 'store' to mean something different to an ObjectBox Store.

Expected behavior

The provided code should execute successfully, deleting the specified 'tiles' from the specified 'stores'.

Actual behavior

The application fatally crashes with the exception shown in the logs below.

Code

As I cannot reproduce the issue myself, I cannot provide an MRE without flutter_map_tile_caching.

However, the original reporter provided an MRE that uses the flutter_map_tile_caching dependency: https://github.com/JaffaKetchup/flutter_map_tile_caching/issues/159#issuecomment-2195800441. It may be that more specific circumstances are required.

I can try to provide more information if possible, but given that I don't think I can reproduce this bug as it is, that's difficult.

The log below is produced on FMTC's example application.

Logs, stack traces

Basic error snippet: DartWorker (177): EXC_BAD_ACCESS (code=1, address=0x12ab68010) iOS Platform Log: pastebin.com/qQCwULRb pubspec.lock (please note that this is a reproduction on my machine, not from the original reporter (see below)): pastebin.com/1vejE0p3

[greenrobot-team]

Thanks for this detailed report! Based on the crash log this looks exactly like a plugin issue on iOS 17 as described in https://github.com/objectbox/objectbox-dart/issues/561#issuecomment-1776592440, which is https://github.com/flutter/flutter/issues/67624.

For the ObjectBox library this was resolved in 2.4.0 though, see the current implementation.

Maybe there is another plugin that is affected?

In any case, the workaround seems to be to always launch the app from Flutter tooling.

Note: labeled this issue with "more info required" so it will auto-close in a few days if there are no follow-up comments.

[JaffaKetchup]

I did create an issue report to Flutter for the path_provider plugin, but they immediately closed it: https://github.com/flutter/flutter/issues/151072 (https://github.com/flutter/flutter/issues/151072#issuecomment-2198808784).

Looking at the implementation of their registration function:

https://github.com/flutter/packages/blob/96f870ecbf3128ec702c98ff519afc76bfeadf17/packages/path_provider/path_provider_foundation/darwin/path_provider_foundation/Sources/path_provider_foundation/PathProviderPlugin.swift#L14-L23

... they don't seem to use the workaround suggested in your linked issue - although they might, I don;t really know enough about plugins:

https://github.com/flutter/flutter/issues/67624#issuecomment-801971172

Could the issue still actually be in path_provider?

---

(Whilst I can't be certain it is exactly the same as OP, the pubspec.lock (https://pastebin.com/1vejE0p3) suggests that they don't really use any other plugins (besides standard core ones). FMTC doesn't make use of any other plugins - and the bug only triggers here.)

[greenrobot-team]

Yes, the path_provider plugin seems to be "affected". But note that even if they apply the recommended workaround, the plugin registration would just not work instead of crash.

I remembered the solution wrong until reading the current state of https://github.com/flutter/flutter/issues/67624 again, to quote:

This workflow [opening the app from the home screen] unfortunately can't work, due to changes in iOS 14. Flutter apps in debug mode on iOS 14+ can only be launched from flutter run, Flutter-aware tools including IDEs, or Xcode

In any case, I think we should add a note about this in our docs.

[JaffaKetchup]

This issue has been confirmed to occur without path_provider involvement: https://pastebin.com/gcMkiGcM. It possibly looks a little different to the original log - but I would've thought it's the same issue, and either way, it's still an issue. It's still a "KERN_INVALID_ADDRESS", but at a different address, and no longer seems to point at registration. See debugging process: https://github.com/JaffaKetchup/flutter_map_tile_caching/issues/159#issuecomment-2233202028.

[greenrobot-team]

@JaffaKetchup Thanks for the update. Unfortunately thread 30 does not have any symbols. And as this is using the ObjectBox Swift library behind the scenes I'm not sure I can symbolize it (we will have to research this more, there is also an open, internal, bug that debug symbols are missing).

Do I remember this right that this can be reproduced (looks like it crashed on an iOS simulator, which I have access to)? Can you give me some coarse steps on how? Maybe I can use an internal debug build of ObjectBox Swift that has symbols to analyze the crash.

[JaffaKetchup]

I believe this can be produced on an emulator using the FMTC app (according to the OP, but I never followed this up) - but I can't verify of course.

The initial steps are the same as #656:

1. Clone the full FMTC repo (the example depends on it): https://github.com/JaffaKetchup/flutter_map_tile_caching

2. Run the example app

3. Follow the screen recording (demoed on Windows)

https://github.com/user-attachments/assets/ec0bb46f-c726-477e-bd9f-6f66b6c0cc3e

I'm not certain which combination of actions is required to produce the issue. The red delete button clears the tiles first, same as the grey delete button, so there shouldn't be any extra info gained by using the red button.

[JaffaKetchup]

Hey, just wondering if there was any chance of a reproduction? I'm happy to provide any more information if I can!

[JaffaKetchup]

It looks like the issue is indeed streaming. I've converted it to use offset and limit instead, as a chunked system, and it now appears to work.

Here's the change I made: JaffaKetchup/flutter_map_tile_caching@ba89ebe (#166).

[greenrobot-team]

Thanks for the update! I had no time to look into this, yet.

Given these changes, I wonder if the Store somehow gets closed while the results are streamed (so as per #379 the new attach API does not prevent this). Something to look into.

[JaffaKetchup]

I guess that might make sense, but then why wouldn't the issue occur on all platforms?

[liuduwei]

HI JaffaKetchup, I'm the user of your lib flutter_map_tile_caching, i encounter this error too, first in my own app use store import/export, and i test the same progress in your lib example, same error, this function is important to me.

I will open a new issue in your repo, and i have two ios phone, can assist you debug this.

[greenrobot-team]

@liuduwei JaffaKetchup said a change was made to the library that works around this issue. Is your project using that version?

On a side note, please do not only share screenshots, also share text. Text can be copied, easily enlarged and is searchable.

In any case, thanks for trying to share more info.

[JaffaKetchup]

Hey @greenrobot-team, The change I made only stabilized one specific method. This isn't that method. It looks similar to the previous issue, possibly to do with streaming in an isolate again? I can't remember off the top of my head so I'd need to check the implementation quickly later.

[JaffaKetchup]

Just FYI, there's been another of the same issue reported on a Discord server. The code is indeed using another streaming technique: https://github.com/JaffaKetchup/flutter_map_tile_caching/blob/b9aaf097a6b49f8d314e6b1aadc0a277ba8cc853/lib/src/backend/impls/objectbox/backend/internal_workers/standard/worker.dart#L1030-L1438.

[greenrobot-team]

Is this still an issue with the 4.3.0 release? The database library received improvements to make sure a Store will wait with closing until all database operations have finished. This theoretically should prevent the EXC_BAD_ACCESS error while results are streaming.

[github-actions[bot]]

Without additional information, we are unfortunately not sure how to resolve this issue. Therefore this issue has been automatically closed. Feel free to comment with additional details and we can re-open this issue.