Crashes on attempting to read realm file from some external drives

[peppy]

SDK and version

SDK : .NET 11.7.0

Potentially related to https://github.com/realm/realm-core/commit/c3ed40bba06bcb76c2ba44646e7f94003e6f3962.

Started in line with an update of realm-dotnet:

https://github.com/ppy/osu/commit/3a224211aa322a0055342f10bd36e0af3c3b078c#diff-11522f45ae2408786975ae807444ce91fea4c4a2cfd21e2f18caf7fa4199412dL38-R38

Observations

• How frequent do the crash occur?

We've had four user reports so far since a new release yesterday. All involve users trying to start the game with a realm file located on an external disk, likely all ExFAT format.

Crash log / stacktrace

Example 1:

2024-03-11 21:33:59 [error]: Realm startup failed with unrecoverable error; starting with a fresh database. A backup of your database has been made.
2024-03-11 21:33:59 [error]: Realms.Exceptions.RealmException: GetFileInformationByHandleEx() failed for 'D:\nochmaosu\client.realm': �p�����[�^�[���Ԉ���Ă��܂��B
2024-03-11 21:33:59 [error]: at Realms.NativeException.ThrowIfNecessary()
2024-03-11 21:33:59 [error]: at Realms.SharedRealmHandle.Open(Configuration configuration)
2024-03-11 21:33:59 [error]: at Realms.RealmConfiguration.CreateHandle(Configuration& configuration)
2024-03-11 21:33:59 [error]: at Realms.RealmConfigurationBase.CreateRealm()
2024-03-11 21:33:59 [error]: at Realms.Realm.GetInstance(RealmConfigurationBase config)
2024-03-11 21:33:59 [error]: at osu.Game.Database.RealmAccess.getRealmInstance()
2024-03-11 21:33:59 [error]: at osu.Game.Database.RealmAccess.prepareFirstRealmAccess()
2024-03-11 21:33:59 [error]: An unhandled error has occurred.

Example 2:

2024-03-12 00:42:21 [error]: Realm startup failed with unrecoverable error; starting with a fresh database. A backup of your database has been made.
2024-03-12 00:42:21 [error]: Realms.Exceptions.RealmException: GetFileInformationByHandleEx() failed for 'F:\osu-lazer\client.realm': The parameter is incorrect.
2024-03-12 00:42:21 [error]: at Realms.NativeException.ThrowIfNecessary()
2024-03-12 00:42:21 [error]: at Realms.SharedRealmHandle.Open(Configuration configuration)
2024-03-12 00:42:21 [error]: at Realms.RealmConfiguration.CreateHandle(Configuration& configuration)
2024-03-12 00:42:21 [error]: at Realms.RealmConfigurationBase.CreateRealm()
2024-03-12 00:42:21 [error]: at Realms.Realm.GetInstance(RealmConfigurationBase config)
2024-03-12 00:42:21 [error]: at osu.Game.Database.RealmAccess.getRealmInstance()
2024-03-12 00:42:21 [error]: at osu.Game.Database.RealmAccess.prepareFirstRealmAccess()
2024-03-12 00:42:21 [error]: An unhandled error has occurred.

In both examples, I've tested the realm file provided by the user and it works fine on my drives (macOS) so the user's database is not at fault.

[sync-by-unito[bot]]

➤ PM Bot commented:

Jira ticket: RCORE-2014

[nirinchev]

Do you know if this only happens on Windows?

[peppy]

@nirinchev So far we've only had reports on windows. I unfortunately don't have an ExFAT formattable drive available to test on macOS today.

As a heads up, we are hotfixing this with a revert from 11.7 to 11.5, as the database files still seem backwards compatible for this version downgrade.

[nirinchev]

So this is upgrading from Core 13.20.1 to 13.26.0. Skimming through the changelog, it appears these 2 changes were related to ExFat filesystems https://github.com/realm/realm-core/pull/6959 and https://github.com/realm/realm-core/pull/7162.

I've also found a similar issue in Go: https://github.com/golang/go/issues/29214 which was fixed by this change: https://go-review.googlesource.com/c/go/+/154377.

@ironage can you take a look since @fealebenpae is off these days and I'm not sure if we have other windows experts around.

[ironage]

There also might be a race condition in InterprocessMutex when a file is reopened while in the process of being closed on another thread

If that is the case, then this might be fixed by https://github.com/realm/realm-core/pull/7448

[peppy]

This has been re-tested by someone on our team and is still a blocking issue, meaning we are currently pinned to <=11.5.0.

[ironage]

Does this happen only when using encryption? If so, it will very likely be fixed by https://github.com/realm/realm-core/pull/7698 since that moves away from using unique file ids altogether.

[peppy]

We are not using realm-level encryption, no.

[Joehuu]

Hey, I'm a contributor in osu! and the crash doesn't seem to happen anymore on https://github.com/realm/realm-dotnet/releases/tag/12.3.0.

[tgoyne]

Good to hear. I think it was incidentally fixed by #7698 because even though we didn't need the file's UniqueID for non-encrypted files we were still reading it for every file opened, and now that code is gone entirely.

[peppy]

Going to close this for now as it seems to be fixed. Thanks!