Question about the commit strategy

[decimad]

Hello, I could not find a source of information about when to place strategic commits. It appeared to me, that unqlite would not commit anything to the file storage until I explicitly commit or close the database. Is there some resource talking about ways to avoid a huge commit on closing the database?

[symisc]

Hi,

The commit strategy is implemented in the file pager.c and largely inspired from the SQLite3 model. Basically, disk commit occurs when:

• Internal memory page cache is filled. You can set a page cache limit via unqlite_config() with a verb set to UNQLITE_CONFIG_MAX_PAGE_CACHE. Once this limit reached, disk commit may occurs.
• Manually commit the transaction via unqlite_commit().
• When you close your database handle via unqlite_close().

[decimad]

Hello, thank your for the reply!

I was very interested in using UNQLITE_CONFIG_MAX_PAGE_CACHE, however I realized that the passed value is completely ignored in the default driver, at least in the latest head I downloaded. Basically I'm trying to use unqlite for the cquery project, but I'm running in the memory wall. Unqlite will buffer 30 gb and more I throw at it in memory, no matter how often I place manual commits. I thought that config setting would do the trick, but following the code paths, it merely sets a member in a structure and is not referenced again. :(

[symisc]

Yes, page caching is delegated to the underlying storage engine which is not required to honor the MAX_PAGE_CACHE instruction. However, you could easily hack the storage engine code now that you have followed the path by just copying the unqlite_close() core code (except closing the handle) when the page cache reaches a maximum limit.

[symisc]

Please update the library to the last version (1.1.9). There was a memory leak in unqlite_commit() that caused internal data not to be freed. This should solve your issue.

[decimad]

Thank you, I'm giving it a try!