[Bug]: Update from V10 to V11 is not keeping existing user database

[Picao84]

Describe the bug 🐞

I have updated to V11 and while it finally works on MAUI on Android, it is not using the same database location used by V10. This is not an app with a cloud backend, so users will lose all their data! Can confirm the appName is exactly the same in both V10 and V11 code.

Step to reproduce

1 - Create an app with V10 and a simple CacheDatabase.UserAccount with a test object with one property with key "test". 2 - Launch the app and update the property value. 2 - Update the app with V11 and load data: it throws KeyNotFoundException. 3 - Reinstall the app with V10 and see that the test object is actually still there.

Reproduction repository

https://github.com/reactiveui/ReactiveUI

Expected behavior

V11 should pick up existing data from V10...

Screenshots 🖼️

No response

IDE

No response

Operating system

No response

Version

No response

Device

No response

ReactiveUI Version

No response

Additional information ℹ️

No response

[glennawatson]

There's examples in the docs of the conversion process and example program in the /src

It's expected behaviour.

[Picao84]

What conversion process? I followed the migration and this is what happens. I see nothing on conversion?

[Picao84]

I see the src folder with example code, but are we expected to have V10 and V11 simultaneously in the same app?

Edit - So to make the conversion I should launch a version of my app with V10 and copy the database to a V11 compatible directory? And then release another version with V11?

This is totally not clear and it's not anywhere I can find in the docs???

[glennawatson]

https://github.com/reactiveui/Akavache/blob/main/docs%2Fmigration-v10-to-v11.md

The data migration part

GitHub

Akavache/docs/migration-v10-to-v11.md at main · reactiveui/Akavache

An asynchronous, persistent key-value store created for writing desktop and mobile applications, based on SQLite3. Akavache is great for both storing important data as well as cached local data tha...

[Picao84]

You even have on the migration guide a section called "Testing your migration" with the comment "Test that old data is still available". There is not mention of a 'conversion' other than src folder?

[Picao84]

https://github.com/reactiveui/Akavache/blob/main/docs%2Fmigration-v10-to-v11.md

The data migration part

GitHub**Akavache/docs/migration-v10-to-v11.md at main · reactiveui/Akavache**An asynchronous, persistent key-value store created for writing desktop and mobile applications, based on SQLite3. Akavache is great for both storing important data as well as cached local data tha...

Yes, I followed that?

GitHub

Akavache/docs/migration-v10-to-v11.md at main · reactiveui/Akavache

An asynchronous, persistent key-value store created for writing desktop and mobile applications, based on SQLite3. Akavache is great for both storing important data as well as cached local data tha...

[glennawatson]

The old serializer is a bson one. So you use that serializer against the path of the old db. Then you can use writer as a example how to convert to v11 more efficient formats.

[Picao84]

Where exactly is that on the docs?

[glennawatson]

Also in terms of location you can manually specify location of the old db with the blobcache. Locations might be slightly different since we are using isolated storage now.

We can probably add a compatibility location helper which gets the old location if it helps.

[Picao84]

Yes, I imagine that would be helpful, otherwise this is really not fully backwards compatible is it?

[glennawatson]

Data Migration

Cross-Serializer Compatibility

V11.1 can read data written by V10.x and different serializers:

// This works! V11.1 can read V10.x data regardless of serializer choice var oldData = await CacheDatabase.UserAccount.GetObject<MyData>("key_from_v10");

Gradual Migration

You can migrate serializers gradually:

// 1. Start with Newtonsoft for compatibility AppBuilder.CreateSplatBuilder() .WithAkavacheCacheDatabase<NewtonsoftBsonSerializer>(/* ... /); // 2. Later, change to System.Text.Json for performance // Old data will still be readable! AppBuilder.CreateSplatBuilder() .WithAkavacheCacheDatabase<SystemJsonSerializer>(/ ... */);

Platform-Specific Migration

.NET MAUI Migration

Before (V10.x):

public partial class App : Application { public App() { InitializeComponent(); BlobCache.ApplicationName = "MyMauiApp"; MainPage = new AppShell(); }

[Picao84]

I had already tried the Bson Serializer by the way. The problem is the location...

[Picao84]

I was able to access the old data through manual instance creation:

public class CustomCacheManager { private readonly IBlobCache _primaryCache; private readonly IBlobCache _secondaryCache;

public CustomCacheManager()
{
    // Create specific cache instances for different purposes
    _primaryCache = new SqliteBlobCache(
        Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "userblobs.db"),
        new SystemJsonSerializer()
    );
    
    _secondaryCache = new InMemoryBlobCache(new SystemJsonSerializer());
}

}

[Picao84]

Could there be any potential downsides to do it this way? I guess I can just retrieve the existing data and then move it to the new instance.

[glennawatson]

Yeah that's a valid way.

@ChrisPulman is also adding a legacy helper which we will release in a couple days.

If they solves your problem while you upgrade your old legacy DB instances it's a valid option and gets you up and running.

[Zack-RI]

@glennawatson @ChrisPulman I noticed there was a change added to add the FileLocationOption - however this isn't available in the latest version of the nuget https://github.com/reactiveui/Akavache/commit/52ffc6060e33ef40851f6e4cd382e0ed91402acf

Are there any plans on releasing this? As things stand now the old caches in our app are inaccessible using V11

This is my findings -

public static MauiAppBuilder ConfigureCacheService(this MauiAppBuilder builder, Action<CacheOptions> cacheOptions)
{
    builder.Services.TryAddSingleton<ICache>(s => RegisterBlobCache(s, cacheOptions));
    return builder;
}

V10:

static CacheService RegisterBlobCache(IServiceProvider s, Action<CacheOptions> optionsDelegate)
{
    optionsDelegate(_options);

    Locator.CurrentMutable.RegisterConstant(_options.JsonSerializerSettings, typeof(JsonSerializerSettings));

    if (string.IsNullOrEmpty(_options.CacheName))
    {
        _options.CacheName = "_cache";
    }

    BlobCache.ApplicationName = _options.CacheName;
    BlobCache.EnsureInitialized();

    var cacheService = new CacheService(BlobCache.LocalMachine);

    return cacheService;
}

This created IBlobCache as a SqlRawPersistentBlobCache object (which isnt even in v11 anymore) and pointed to "/data/user/0/package.name/cache/blobs.db" (Where is ApplicationName used here?? Looks like it just ignores it)

V11:

static CacheService RegisterBlobCache(IServiceProvider s, Action<CacheOptions> optionsDelegate)
{
    optionsDelegate(_options);

    AppLocator.CurrentMutable.RegisterConstant(_options.JsonSerializerSettings, typeof(JsonSerializerSettings));
   
    if (string.IsNullOrEmpty(_options.CacheName))
    {
        _options.CacheName = "_cache";
    }

    AppBuilder.CreateSplatBuilder()
              .WithAkavacheCacheDatabase<NewtonsoftSerializer>(cacheBuilder =>
                                                                       cacheBuilder.WithApplicationName(_options.CacheName)
                                                                           .WithSqliteProvider()
                                                                           .UseForcedDateTimeKind(DateTimeKind.Utc)
                                                                           .WithSqliteDefaults());

    var cacheService = new CacheService(CacheDatabase.LocalMachine);

    return cacheService;
}

This creates it as a SqliteBlobCache object pointing to "/data/user/0/package.name/files/.config/.isolated-storage/ApplicationName/LocalMachine/LocalMachine.db"

Also in the code it would be good to add a flag that if it cant find the db, check the legacy location first. If it's there then use this one, else create it in the new location

[Qonstrukt]

I was also very surprised to find that my old data doesn't seem to be found after upgrading to v11 and going through the whole migration document.

Overview of Changes

Akavache V11.1 introduces significant architectural improvements while maintaining backward compatibility with your data. The main changes are in how you initialize and configure Akavache, not in the core API you use daily.

Breaking Changes

1. Initialization Method: The BlobCache.ApplicationName and Registrations.Start() methods are replaced with the > builder pattern
2. Package Structure: Akavache is now split into multiple packages
3. Serializer Registration: Must explicitly register a serializer before use

This doesn't mention anything about losing data, or data migration steps.

I've followed all the Migration Steps:

1. Update Package References ✅
2. Update Initialization Code ✅
3. Migration Helper ❓

Playing Devil's advocate here: What's that for, what does it do, why is it mentioned here? It just seems to wrap the new initialisation code in a single method, so it doesn't seem useful on top of the other steps. Also, in all of a sudden it references DatabaseCache instead of Splat.Builder, so I find this step very confusing.

Then to be sure, I checked the Detailed Migration Scenarios.

1. Serializer Migration: I choose to stick with Json.NET for now, so I just use that serializer ✅
2. Data Migration literally says this:

Cross-Serializer Compatibility

V11.1 can read data written by V10.x and different serializers:

// This works! V11.1 can read V10.x data regardless of serializer choice var oldData = await CacheDatabase.UserAccount.GetObject<MyData>("key_from_v10");

So I should be done? Except I'm not, none of my old data is showing up.

Maybe kind of a long-winding write down of events, but that's exactly my point. The migration guide is quite long, contains confusing supplementary(?) information. And misses critical information that's important to anyone updating to this version.

Might've just been an oversight, but from the reactions to @Picao84 I got the feeling he wasn't understood, and that it should all be very clear? But the documentation definitely is not clear about this.

Thanks as always. I'll gladly update the documentation myself as I've done for ReactiveUI in the past, but even from @Zack-RI's post it's not quite clear to me how this would work out for everyone, and all the different caches that exist.

[glennawatson]

@Qonstrukt did you end up getting your data loading in the end or are you still blocked?

Yeah I tried to separate out the documentation a bit (with a bit of AI help but lots of human double checking so its possible something snuck out).

[Picao84]

@Qonstrukt` I was able to retrieve V10 data on iOS this way:

`var oldCache = new SqliteBlobCache(Path.Combine(CreateAppDirectory(NSSearchPathDirectory.DocumentDirectory, CacheDatabase.ApplicationName), "userblobs.db"), new NewtonsoftSerializer() );

private static string CreateAppDirectory(NSSearchPathDirectory targetDir, string applicationName, string subDir = "BlobCache") { using var fm = new NSFileManager();

    var url = fm.GetUrl(targetDir, NSSearchPathDomain.All, null, true, out _) ?? throw new DirectoryNotFoundException();
    var rp = url.RelativePath ?? throw new DirectoryNotFoundException();
    var ret = Path.Combine(rp, ".config", applicationName, subDir);
    
    return ret;
}`

For Android this worked, like I mentioned above:

public CustomCacheManager() { // Create specific cache instances for different purposes _primaryCache = new SqliteBlobCache( Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "userblobs.db"), new SystemJsonSerializer() );

_secondaryCache = new InMemoryBlobCache(new SystemJsonSerializer());

}

[Qonstrukt]

@glennawatson no, unfortunately not. I don't have the time to write a migration at this point and find out where old data was vs where it's now stored myself. I was hoping to do some dependency upgrades on a project, but I guess it's gonna take some more time.

I understand the trickiness of good documentation. Just wanted to warn that this might catch some people off guard.

[EmilAlipiev]

so previous v10 and v11 db location is different, is this correct? if that is the case, how does automatic data migration makes sense without resolving compatibility issue? in the documentation i cannot see any solution for this. I have just tested on Android and it also doesnt work for me. btw thanks for the great effort and changes. everything else seems to be robust.

[EmilAlipiev]

@ChrisPulman is also adding a legacy helper which we will release in a couple days.

new path:

old path:

has this legacy helper already released? how it should help us?