Heap Corruption and Crashes in WinRT due to Pinvoke

[legistek]

I've encountered what appears to be a bug in the Windows Rutnime that can cause heap corruption - and therefore unpredictable crashes - when using SQLite.Net PCL in a Runtime app.

In my case I don't use the SQLite VSIX extension through Nuget but rather incorporate sqlite.c directly into a C++ Windows Runtime Extension DLL that also has other stuff I need written in C++. Don't know if that makes a difference, or why it should.

Anyway, I found that I would get random heap corruption and System.ExecutionEngineException errors that were all but untraceable. Through belabored testing and elimination, I finally tracked it down to the API class used to invoke the SQLite3 DLL. Something bad seems to be happening in the marshaling of strings to LPSTRs and LPWSTRs. Unfortunately I was never able to modify the .CS marshaling behavior to avoid the crashes. Instead, I wrote a helper ref class in my C++ module that mimics the behavior of the CS API class but handles the conversion between Strings and LPSTRs and LPWSTRs in its own code. And guess what - no more crashes.

Again I don't know if - or why - it could be related to my using sqlite.c rather than the VSIX extension. And I can find nothing wrong with the API code file. Therefore I think it is a bug in the CLR or Windows Runtime itself. My recommended fix is to implement the helper class in C++ as I did. Because you cannot exchange IntPtr's between C# and C++ with WinRT, nor pass "out" parameters, a little more creativity is necessary, but it certainly can be done.

Just thought I'd share my experience in case anyone has found the same thing.

[oysteinkrog]

I really doubt it's a bug in the runtime; I've written a lot of custom pinvoke and there are lots of things that can go wrong. I recommend you try to create a minimal testcase and then start changing things. Make sure you use the correct calling convention and alignment/packing. If the issue is only related to strings, you are probably assuming things about how pinvoke and strings work that are wrong. If I remember correctly pinvoke with strings directly in the binding have assumptions built in about how the string memory is allocated (and can be free'd).

[legistek]

Hey Øystein- Thanks for your reply. I actually was using SQLite.NET PCL out of the box with no changes and was getting Pinvoke errors. In fact I removed ALL C++ dependence from my code other than SQLite.NET->Sqlite3.dll. Perhaps it’s a bug in Sqlite itself (certainly not out of the question). I will attempt test cases that do not rely on SQLite at all (difficult since my app is extremely dependent on it) to confirm that the problem lies there.

Peter

From: Øystein Krog [mailto:[email protected]] Sent: Saturday, August 8, 2015 12:22 PM To: oysteinkrog/SQLite.Net-PCL [email protected] Cc: Peter N. Moore [email protected] Subject: Re: [SQLite.Net-PCL] Heap Corruption and Crashes in WinRT due to Pinvoke (#205)

I really doubt it's a bug in the runtime; I've written a lot of custom pinvoke and there are lots of things that can go wrong. I recommend you try to create a minimal testcase and then start changing things. Make sure you use the correct calling convention and alignment/packing. If the issues is only related to strings, you are probably assuming things about how pinvoke and strings work that are wrong. If I remember correctly pinvoke with strings directly in the binding have assumptions built in about how the string memory is allocated (and can be free'd).

— Reply to this email directly or view it on GitHubhttps://github.com/oysteinkrog/SQLite.Net-PCL/issues/205#issuecomment-129014209.

[DennisSc]

Well... I (sort of) can confirm this issue... I think.

Unhandled exception at 0x775B914B (ntdll.dll) in App1.UWP.exe: 0xC0000374: A heap has been corrupted (parameters: 0x775EEB70).

this I get from sqlite PCL in a w10 universal app, and after pulling my hairs for several days, refactoring, code review, rewriting everything sqlite related completely as "async" methods, (re-)using only one (locking) sqlite connection throughout the whole app, or starting (and closing) separate connections in each function - doesn't matter what I try, I end up with above or similar errors when I "stress" my app (not really much, a few queries per second).

The visual studio debug console also shows me a Critical error detected c0000374.

Here's my current failing code (when it does fail, it breaks on the line "static Func"...,, if this helps):

 class SQLfactory
    {
       static string path = Path.Combine(Windows.Storage.ApplicationData.Current.LocalFolder.Path, "deviceDB.sqlite");
        static Func<SQLiteConnectionWithLock> connectionFactory = new Func<SQLiteConnectionWithLock>(()=>new SQLiteConnectionWithLock(new SQLite.Net.Platform.WinRT.SQLitePlatformWinRT(), new SQLiteConnectionString(path, false)));
        static SQLiteAsyncConnection conn = new SQLiteAsyncConnection(connectionFactory);

        public SQLfactory()
            {
                 async void createTable() { CreateTablesResult x = await conn.CreateTableAsync<Device>(); }
            }

and then later on, using this SQLiteAsyncConnection object in various other methods of the same class, like:

       public static async Task<IEnumerable<Device>> GetDevicesFromDBAsync()
        {
            string sql = "SELECT * FROM Device";
            var result = await conn.QueryAsync<Device>(sql, "");
            return result;
        }

As said, when such methods are called often, the app crashes, reproducibly. What am I doing wrong? Or same issue as described above?

thx

[eriksimonic]

Hi. Did you manage to solve this problem, as I am experiencing a similar scenario. Or lets say it it is identical to your.

[DennisSc]

Well, yes and no - I did manage to "solve" that problem, sort of, in a rather pragmatic way....

I switched to using Microsofts own sqlite implementation (https://github.com/aspnet/Microsoft.Data.Sqlite) and haven't had any problems since.

best regards

[zstarkebaumcalico]

It looks like this issue could be related to creating a new SQLitePlatformWinRT more that once in the application. According to the documentation for Sqlite3. These are the limitiations of setting the temp directory for sqlite3 (which is what the constructor for SQLitePlatformWinRT does).

It is not safe to read or modify this variable in more than one thread at a time. It is not safe to read or modify this variable if a database connection is being used at the same time in a separate thread. It is intended that this variable be set once as part of process initialization and before any SQLite interface routines have been called and that this variable remain unchanged thereafter. https://www.sqlite.org/c3ref/temp_directory.html

So maybe do something like this:

class SQLfactory
{
    static SQLitePlatformWinRT _platform = new SQLitePlatformWinRT(); 

    public SQLiteConnectionWithLock CreateConnection(string path) 
    {
        return new SQLiteConnectionWithLock(_platform, new SQLiteConnectionString(path, false));
    }
}