[BUG] Out of memory - hijacked by v9 release info

[ozzy1873]

What is the bug?

I received an error at startup: IsarError: Cannot open Environment: MdbxError(12) Out of memory

What is the expected behaviour?

No crash

How can we reproduce this issue?

No response

Do you have a potential solution?

No response

Can you provide any other information?

The phone is a Samsung S5 (SM-G900F)

Platforms Affected

Android

Severity

Fatal: Causes the application to crash

Frequency

Consistently: Always occurs at the same time and location

Requirements

• [X] I agree to follow this project's Code of Conduct
• [X] I have checked for known issues in the GitHub tracker and on the docs site
• [X] I have completed the pre-report checks listed on the docs site
• [X] If I am using FMTC in a proprietary project, I have read and understood this important legal notice

[JaffaKetchup]

Hey @ozzy1873,

How many tiles are in your store, and now large is the store?

You can try decreasing the databaseMaxSize configuration in the FMTC settings. Alternatively, using multiple stores might help.

Isar depends on MDBX, which is fairly memory intensive. You might think storage and memory aren't linked, but they are in Isar. You may want to open an issue with Isar to see if there's anything they can do about this - I've had another report privately. Additionally, as far as I understand, Isar should be using virtual memory more than real memory.

[ozzy1873]

Hi Luka,

There are about 3000 tiles in the store and the app is migrating from v6 to v7. Is there a way to catch the error and reset the cache, rather than crashing the app?

Thanks! John

From: Luka S @.> Sent: Wednesday, February 22, 2023 10:46 AM To: JaffaKetchup/flutter_map_tile_caching @.> Cc: john osborntech.com @.>; Mention @.> Subject: Re: [JaffaKetchup/flutter_map_tile_caching] [BUG] Out of memory (Issue #105)

Hey @ozzy1873https://github.com/ozzy1873,

How many tiles are in your store, and now large is the store?

You can try increasing the databaseMaxSize configuration in the FMTC settings. Alternatively, using multiple stores might help.

Isar depends on MDBX, which is fairly memory intensive. You may want to open an issue with Isar to see if there's anything they can do about this - I've had another report privately.

— Reply to this email directly, view it on GitHubhttps://github.com/JaffaKetchup/flutter_map_tile_caching/issues/105#issuecomment-1440403577, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACCL55ACTVJMQBGOQDOLVDTWYY7FNANCNFSM6AAAAAAVEHJFDQ. You are receiving this because you were mentioned.Message ID: @.@.>>

[JaffaKetchup]

I've edited my comment above, please check there.

Unfortunately there's no way to catch this. It seems there are some stability issues, and I may have to workaround them.

Out of interest for the workaround, do you use import/export functionality?

[JaffaKetchup]

Can you also try running a clean version of v7 (no migration), to see if the issue occurs then? There is a possibility the migrator code is causing issues, having looked around Isar's tracker.

[ozzy1873]

I am not using import/export.

[JaffaKetchup]

Can you also try running a clean version of v7 (no migration), to see if the issue occurs then? There is a possibility the migrator code is causing issues, having looked around Isar's tracker.

Can you also try this please?

[JaffaKetchup]

@ozzy1873 If it is indeed an issue with the migrator, I've implemented a fix that might resolve it. Can you check the 'v7.1.3' branch please, it might fix this issue? To depend on a GitHub repository branch, please see https://fmtc.jaffaketchup.dev/get-started/installation#from-github.com, and ref: v7.1.3 below the url field.

[dumabg]

It occurs the same to me. First time using your libraries, so, new code. Using import/export.

I've run the example and do an export. The file generated has only 198 tiles, only 10MB:

export.fmtc.zip

I've copied this generated file on the assets directory and created this code:

maps_offline_service.dart.zip

Debugging on Android, the FMTCImportSharingModule.manual method throws the exception on Isar.open call:

FMTCRegistry.instance.storeDatabases[id] = await Isar.open( [DbStoreDescriptorSchema, DbTileSchema, DbMetadataSchema], name: id.toString(), directory: FMTC.instance.rootDirectory.directory.path, maxSizeMiB: FMTC.instance.settings.databaseMaxSize, compactOnLaunch: FMTC.instance.settings.databaseCompactCondition, );

[ozzy1873]

Unfortunately, it will be a few days before I can try out the new version. Also, this is the phone of one of my beta testers, so I do not have direct access to it. Can you think of a way that an emulator could be used to test this scenario?

[JaffaKetchup]

@ozzy1873 Any x64 emulator should work (x86 doesn't). Try to get the specs the same as the real device.

@dumabg Can you say how much memory your testing device has please? I'd like to try and recreate this, as I can't on my device (and obviously my Windows machine doesn't struggle at all).

[ozzy1873]

The problem with the emulator is setting up the initial conditions so that the v6->v7 upgrade happens on existing files.

[JaffaKetchup]

That's a little more difficult I guess. Can you just test without any migration for me, just to isolate the issue? I don't mind waiting a few days. (Transferring the files could be done with the import/export functionality, but don't worry about setting that up just for this).

[dumabg]

@dumabg Can you say how much memory your testing device has please? I'd like to try and recreate this, as I can't on my device (and obviously my Windows machine doesn't struggle at all).

3GB

[JaffaKetchup]

@dumabg Thanks. Just to confirm, you're not affected by the migration code? And are you importing/exporting the store when the error occurs?

[dumabg]

@dumabg Thanks. Just to confirm, you're not affected by the migration code? And are you importing/exporting the store when the error occurs?

First time using the library. New code. These are the versions on pubspec:

flutter_map_tile_caching: ^7.1.2
fmtc_plus_sharing: ^7.0.0

And yes, I'm using the FMTCImportSharingModule.manual method. See the sample attached on the previous post.

[JaffaKetchup]

@dumabg When testing with the example app, I can reproduce a different issue on my Windows machine, and on a Pixel 2 x64 2GB RAM emulator. If the store file is imported without the name format 'export_<name>.fmtc', the store doesn't get properly don't seem to work right: this is a bug, thanks for reporting. In your case, 'export.fmtc' doesn't work correctly. I'll have to fix this, unfortunatley likely with a breaking change, looking at the situation. However, after renaming the file to 'export_catetinho.fmtc', it seems to work correctly. Can you post your console output, and if possible, a screen-recording? Your code looks OK, although I haven't tested it. It does seem a little unusual though to be getting the FMTC store from assets. Thanks for all the help so far.

(My emulator config:)

[dumabg]

I've changed the code and put "catetinho" instead of "map", but continues with the same error:

Debugger screenshoot, stopped on the problematic line. You can see that the input file has the name export_catetinho.fmtc:

After this point, appears this on the debug console:

I/flutter (27813): IsarError: Cannot open Environment: MdbxError (12): Out of memory

W/isarworker(27813): type=1400 audit(0.0:5077): avc: denied { getattr } for uid=10237 path="/persist" dev="mmcblk0p30" ino=2 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:persist_file:s0 tclass=dir permissive=0 I/chatty (27813): uid=10237(catetinho.abdi.app) identical 16 lines W/isarworker(27813): type=1400 audit(0.0:5094): avc: denied { getattr } for uid=10237 path="/persist" dev="mmcblk0p30" ino=2 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:persist_file:s0 tclass=dir permissive=0 I/flutter (27813): IsarError: Cannot open Environment: MdbxError (12): Out of memory

El 22/2/23 a las 17:55, Luka S escribió:

@dumabg https://github.com/dumabg When testing with the example app, I can reproduce a /different/ issue on my Windows machine, and on a Pixel 2 x64 2GB RAM emulator. If the store file is imported without the name format 'export_.fmtc', the store doesn't get properly don't seem to work right: this is a bug, thanks for reporting. In your case, 'export.fmtc' doesn't work correctly. I'll have to fix this, unfortunatley likely with a breaking change, looking at the situation. However, after renaming the file to 'export_catetinho.fmtc', it seems to work correctly. Can you post your console output, and if possible, a screen-recording? Your code looks OK, although I haven't tested it. It does seem a little unusual though to be getting the FMTC store from assets. Thanks for all the help so far.

— Reply to this email directly, view it on GitHub https://github.com/JaffaKetchup/flutter_map_tile_caching/issues/105#issuecomment-1440784741, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIMXZOEH43GW46MVIQV2ZRTWYZ4NTANCNFSM6AAAAAAVEHJFDQ. You are receiving this because you were mentioned.Message ID: @.***>

[JaffaKetchup]

@dumabg Thanks for the info. Tomorrow I'll give it a go on an older physical device of mine to see if that helps me reproduce the issue.

[JaffaKetchup]

To everyone subscribed. I'm currently planning on ways to fix this issue, as a private contact has also confirmed that they recieve this error. Unfortunatley, I am rather busy at the moment. At the moment, my plan is to offer an option to dump tiles onto the filesystem, as in pre-v7. There would be no seperation of them, and choosing this method would disable the import/export functionality on that store. Isar would still be required to manage the tiles and store information, but it would only store IDs/references, instead of the tile bytes themseleves, significantly reducing the strain on Isar. This does have a few catches, that I'd need to work out before implementation. Let me know if this sounds good to you guys. Note that this will be v8.0.0.

[ozzy1873]

Hi Luka,

Storing the tiles as separate files outside the database sounds like a good idea. There can be a lot of cached files, so this would reduce the load on the database.

Regards, John

From: Luka S @.> Sent: Wednesday, March 1, 2023 2:26 PM To: JaffaKetchup/flutter_map_tile_caching @.> Cc: john osborntech.com @.>; Mention @.> Subject: Re: [JaffaKetchup/flutter_map_tile_caching] [BUG] Out of memory (Issue #105)

To everyone subscribed. I'm currently planning on ways to fix this issue, as a private contact has also confirmed that they recieve this error. Unfortunatley, I am rather busy at the moment. At the moment, my plan is to offer an option to dump tiles onto the filesystem, as in pre-v7. There would be no seperation of them, and choosing this method would disable the import/export functionality on that store. Isar would still be required to manage the tiles and store information, but it would only store IDs/references, instead of the tile bytes themseleves, significantly reducing the strain on Isar. This does have a few catches, that I'd need to work out before implementation. Let me know if this sounds good to you guys. Note that this will be v7.2.0.

— Reply to this email directly, view it on GitHubhttps://github.com/JaffaKetchup/flutter_map_tile_caching/issues/105#issuecomment-1450800894, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACCL55ENLVSF4RJYEW7NUHLWZ6WFNANCNFSM6AAAAAAVEHJFDQ. You are receiving this because you were mentioned.Message ID: @.@.>>

[JaffaKetchup]

@ozzy1873 @dumabg

I've been having a think about some of the issues with storing tiles in a dump. Whilst many of them could be solved, it would become incredibly complex. These are the main issues:

• No fine-grained control of stores
  • Inability to delete individual stores
    This would require sorting through all stores and their tiles to find the correct tile IDs to delete: too slow.
  • Inability to watch individual stores IO watching watches an entire directory or a single file. Starting multiple watchers is extremely costly, especially on some platforms.
• Cannot get size statistics (length is available)
  This is just way to slow on IO, and trying to store sizes in the databases would be relatively slow to reach sums as well. Even a solution where storing and deleting tiles updates one value would not be acceptable: guaranteeing sync is impossible (at any reasonable speed/frequency).
• No store importing/exporting
  This would require sorting through all stores and their tiles to find the correct tile IDs to add to and archive: too slow.
• Significant performance reductions throughout
  IO operations on multiple files is extremely slow, even on a superfast SSD. Creating, deleting, getting metadata (size, time, etc.), all takes a really long time on 30K files.

Potential, one directory could be used per store, similar to pre-v7, but this has it's issues:

• Statistic getting would be very slow
  Pre-v7, statistic caching was relied on to get some sort of acceptable performance. I will not be implementing this again.
• Store importing/exporting would be very slow
  Pre-v7, stores had to be archived (zipped) before sharing: too slow. I will not be implementing this again.

You can see why I migrated to Isar :).

I haven't had too many reports about this issue. In total, including both of you, I've only had 3 reports. In terms of my luck in reproducing, I haven't been able to (with the example app). I've gone all the way back to an Android 6 device (Motorola Moto G 2015) with 1GB LPDDR3 RAM and ~300MB RAM remaining when testing. At most, with ~8000 tiles (~78 MiB), I've only been able to produce a memory consumption of ~110MB: which is a very reasonable amount.

I have recently made some improvements in 7.2.0 that might help some things. Please try running this just to double check if you run into issues.

Please let me know your opinions and thoughts.

[tratteo]

Hello, I am getting the same error IsarError: Cannot open Environment: MdbxError (12): Out of memory running on a Pixel 6, API 27.

• Maps downloaded are very small.

• I am using different stores to cache different maps: var store = FMTC.instance(activityId);

In my case the error happens when I try to access a second store after a first one has already been accessed. Indeed if I only access the first store, the error is not thrown.

However, testing on a Pixel 6 API 25 (Android Nougat 7) the error was thrown directly at await FlutterMapTileCaching.initialise()

Configuration for the Pixel 6 API 27:

[JaffaKetchup]

Hi @tratteo, and thanks for your report. You may have just caught onto something with the 'using multiple stores'. See https://github.com/isar/isar/discussions/738#discussioncomment-3870228. I've made an offer to the maintainer of a bounty (https://github.com/isar/isar/discussions/738#discussioncomment-5294617) to resolve a few issues, so I'll see if he gets back after that. Note that money for bounties can only come out of donations and alternative license payments, so please consider donating if you can!

[tratteo]

Thanks for the fast reply, I will see what I can do 😅

By the way, if the problem is related to multiple stores, I find it strange that it never happened on API 28+

[JaffaKetchup]

@tratteo That's an interesting point about the API levels. The original device produce to report this issue was a Samsung S5, which appears to ship with Android 5: below API level 28. However, I tested on a device below API level 28 (albeit with only one store), and there was no issue. So it may not affect all devices. I'll test on that device again with multiple stores, to see if that affects things.

(Don't feel pressured into donating, please only do so if you've gotten FMTC working and you want to :))

[tratteo]

Thanks, let me know if you encounter the problem while testing with multiple stores

[tratteo]

Just to throw some more information: I tested again with an emulator of Pixel 6 API 27. I am doing some tests, decreasing the databaseMaxSize down as 64 Mib seems to allow the creation of multiple databases, at least up to 6 stores. To be honest 64 Mib should be more than enough for our purpose but it is assured that it is not something normal on the Isar's side

Edit: Downloading an area of around 8Km in radius (1500 tiles) crashes the app with databaseMaxSize=64, so i guess it requires a bigger db storage somehow.

PS. I noticed that FMTC.instance.rootDirectory.stats.rootSize returns the result in Mib instead of Kib as the division takes places two times:

double get rootSize =>
      storesAvailable.map((e) => e.stats.storeSize).sum / 1024;

and

double get storeSize => _db.getSizeSync(includeIndexes: true) / 1024;

[JaffaKetchup]

@tratteo Thanks for that bug report at the bottom there. If you get time, can you please open a seperate issue?

Downloading an area of around 8Km in radius (1500 tiles) crashes the app with databaseMaxSize=64, so i guess it requires a bigger db storage somehow.

Is there a specific error message? 64MiB is fairly small for a tile store, so I wouldn't be surprised if it was that.

I haven't gotten around to that second round of testing yet, I'll try to do that today or over the weekend.

[tratteo]

At the moment I was not able to reproduce the problem. Downloading tiles for around 40 Mib does not crash the app with a db size of 64, this is indeed weird, as it did cause crash before.

I am noticing something else tho, due to some test I was doing with the package, whenever exceptions are thrown, the size of the store increases enormously. For example when using a cached tile provider for visualising the map offline and moving out of the cached zone, I get tons of these errors:

Whenever this happens, the size of the root folder increases. As an example, just moving a second out of the cached area, increases the root store size of more than 10 Mib. Are these errors and the size increment normal?

[JaffaKetchup]

Hi @tratteo, I can't seem to reproduce the growing database size issue. However, Isar does suggest that long running transactions can cause database growth, so this may be an issue if errors are throwing, but not exiting transactions. You can the adjust databaseCompactConditions, however, which is run on every initialisation. This should remove any of that consumed space.