API Sketches for Transactions

[anaisbetts]

API Sketches for My Sweetheart the Drunk Transaction

• The mindblowing idea is, you simply cannot access normal read/write operations outside of a transaction. They're not even on the IBlobCache object. Instead, you create a transaction and the returned transaction object has all of the methods you're used to. You can't Hold It Wrong™.

Old:

await BlobCache.LocalMachine.InsertObject("Foo", bar);
await BlobCache.LocalMachine.GetObjectAsync<Baz>("Bamf");

Perhaps New:

// Return value must:
// * Return the value of the transaction at the end or be Unit
// * Have a Cancel() method to cancel the transaction
await BlobCache.LocalMachine.InTransaction(cache => async {
    await cache.InsertObject("Foo", Bar);
    return await cache.GetObjectAsync<Baz>("Bamf");
});

or perhaps just:

await BlobCache.LocalMachine.InSyncTransaction(cache => {
    // These all run in order
    cache.InsertObject("Foo", Bar);
    cache.InsertObject("Baz", Bamf);

    return cache.GetObject<Bard>("Shakespeare");
});

Maybe both? There should also be a method for one-offs:

// I hate the name "SingleOp", find something better
await BlobCache.LocalMachine.SingleOp().GetAllObjects<Bamf>();

or times when you can't jam everything into a single Lambda function:

var transaction = BlobCache.LocalMachine.BeginTransaction();

await transaction.GetObjectAsync<Bar>("Foo");

transaction.Dispose();   // Finishes the transaction
//or
transaction.Cancel();    // Aborts the transaction.

Open Questions

• Are operations inside a transaction guaranteed to run sequentially? Should they be?
• Do transactions only apply to writes, or does it guarantee a consistent read "view of the world".

[jspahrsummers]

There should also be a method for one-offs:

I agree if you can only get objects as a one-off operation. IMO, you should never be able to write outside of a transaction, because there's a high likelihood that you're doing it wrong.

or times when you can't jam everything into a single Lambda function:

Like when?

Are operations inside a transaction guaranteed to run sequentially? Should they be?

Absolutely. If you want out-of-order execution, use separate transactions.

Do transactions only apply to writes, or does it guarantee a consistent read "view of the world".

If you think of the backing store as a serial queue, a transaction (as a whole) should be a blocking unit of work on that queue.

You could probably optimize that in certain ways, but I do think it's important that you have a 100% consistent view of the world within anything called a "transaction." Or, if that's simply not possible, support retrying.

[choudeshell]

Are operations inside a transaction guaranteed to run sequentially? Should they be?

Yes. Async patterns are fully compatible with sequential events. Although it may cause problems when Async patterns are purposely executed on separate threads -- no longer guaranteeing sequential actions. They should also be atomic.

Maybe both? There should also be a method for one-offs:

I believe a better way of doing transactions and the one-offs is:

using(var cache = BlobCache.LocalMachine.TransactionFactory.BeginTransaction())
{
              cache.InsertObject("Foo", Bar);
              cache.InsertObject("Baz", Bamf);
              cache.Complete();
}

BlobCache.LocalMachine.TransactionFactory.BeginTransaction(cache=>
{
              cache.InsertObject("Foo", Bar);
              cache.InsertObject("Baz", Bamf);
              cache.Complete();
});

cache.InsertObject("Foo", Bar);
cache.InsertObject("Baz", Bamf);

A simple facade via interfaces should be enough to accomplish this. The beauty is that you call the same methods regardless of if it is in a transaction or not (leaving out the call to Complete when it is not in a transaction).

EDIT

On second thought -- another way to do transactions vs one-offs is that one-offs always create an implicit/backing transaction. Again a facade via interfaces should accomplish this.

[choudeshell]

Do transactions only apply to writes, or does it guarantee a consistent read "view of the world".

This depends on when you commit the objects to the store. If you explicitly have to commit (via some method) then yes, this would also 'guarantee' a consistent read -- ie. no 'dirty' reads. On the other hand since there is no explicit locking, this would truly end up depending on the backing store.

[kellabyte]

Are operations inside a transaction guaranteed to run sequentially? Should they be?

I agree with @jspahrsummers that operations inside a transaction should be guaranteed to be in order. Most transactional databases work this way, so when people see your code they will expect this behaviour.

What would a failed transaction look like? It's not clear to me in the examples what the failure semantics are.

Do transactions only apply to writes, or does it guarantee a consistent read "view of the world".

In an ideal world, all transactions include reads and writes and are truly ACID with the strictest isolation level (serializable) but these guarantees come at a cost. If this is the case then reads cannot see intermediate results of writes in any circumstance. I don't know the goals of Akavache so I'm not sure if ACID using the strictest guarantees makes sense or not. You can relax guarantees and go with isolation levels like repeatable reads, read committed, or even the loosest - read uncommitted. Just as some background info, even though many RDBMS's are ACID databases, very few use serializable isolation as the default. For example, SQL Server uses read committed as the default so it's not as correct as it could be. Good performance for mostly correct data. Calculated trade-off's I suppose :)

The answer to this probably lies in what kinds of operations people are going to be doing with Akavache. If you have concurrent transactions competing on a piece of data and calculating some outcome, there's risk it will be using dirty reads from another partially completed transaction and create incorrect output for example. If that situation is really low risk, then maybe you can make do with less guarantees.

I'm not sure if I helped any but that's what came to mind :P

[kellabyte]

On second thought -- another way to do transactions vs one-offs is that one-offs always create an implicit/backing transaction. Again a facade via interfaces should accomplish this.

This is exactly what a lot of databases do internally. It has to do this inside the transaction coordinator anyway because it still has to isolate other in-progress transactions from the one-off.

[danielmarbach]

Hy paul

See my suggestions here:

http://www.planetgeek.ch/2012/05/05/what-is-that-all-about-the-repository-anti-pattern/

Public interface IUnitOfWork {
   IUnitOfWorkScope Start();
}

public interface IUnitOfWorkScope : IDisposable
{
    void Commit();
    void Rollback();
    ...
}

[kellabyte]

Another thing to consider is batch persisting. It's perfectly acceptable to batch writes to the storage engine to gain write performance as long as when a failure occurs you properly fail the related transactions. If it fails though, it has to fail as a unit and not persist half of the transactions.

[anaisbetts]

I agree if you can only get objects as a one-off operation. IMO, you should never be able to write outside of a transaction, because there's a high likelihood that you're doing it wrong.

So, doing the one-off is just shorthand for creating a one-item transaction.

Absolutely. If you want out-of-order execution, use separate transactions.

Legit

[jspahrsummers]

So, doing the one-off is just shorthand for creating a one-item transaction.

Yes, though I meant an implicit transaction where writing is disallowed. If you see this floating in the middle of nowhere:

await BlobCache.LocalMachine.InsertObject("Foo", bar);

… what are the chances that it properly handles concurrency? There are certainly valid cases for doing that (e.g., initial insertion into the cache), but there are many more invalid cases that are easy to miss.

[anaisbetts]

I don't know the goals of Akavache so I'm not sure if ACID using the strictest guarantees makes sense or not. You can relax guarantees and go with isolation levels like repeatable reads, read committed, or even the loosest - read uncommitted

Yeah, these are the things I'm trying to mull over - what guarantees do you need to make it sane to write desktop / mobile apps against, vs. what guarantees a full RDBMS has that you don't really need when writing a desktop app

[DennisWelu]

(Four years later...)

Is there still no way to do a transaction of any sort? My interest is in at least being able to put 2 edits in a transaction - one insert, and one in a corresponding "log". I'm able to reach into the SQLiteConnection and issue the begin/commit/rollback, but the insert operations seem to be hanging and I'm wondering if it's due to one thread locking another in the sequence.

On the larger discussion I agree I wouldn't expect a robust RDBMS style implementation, but some basic operations like this would be very nice!

[DennisWelu]

Best design aside, here is something I did to provide manual transaction control in a quick-and-dirty fashion against the current code: https://github.com/DennisWelu/Akavache/commit/a7122d0ee25e175961e338ac29d278952f57dd59.

The basic idea is to have the OperationQueue hang on to the SQLConnection it's already getting and then during ProcessItems if a transaction is already started it doesn't wrap everything in a transaction automatically. It's up to the caller to manage... The downside is you take on the responsibility of making sure your transactions are ended one way or another come success or exceptions.

Access to the connection is then directly available on an IBlobCache by getting it from its implementation in SQLitePersistentBlobCache:

    SQLiteConnection Connection => ((SQLitePersistentBlobCache) Cache).Connection;

And transactional control can be done something like this:

        try
        {
            Connection.BeginTransaction();
            //...do some things in order, (a)waiting completion of each
            Connection.Commit();
        }
        catch (Exception)
        {
            Connection.Rollback();
            throw;
        }

FWIW!

[DennisWelu]

(Two years (note quite) later...)

Not much activity on this thread. I'm assuming there hasn't been much interest in transaction support. But I can say that I've been using the changes I noted above quite a bit to do simple "2 operation" transactions and it's worked out well. For lack of a better design if there were more interest voiced I could submit a PR for the changes. There isn't a lot of "new code" to make it happen but it does affect the OperationQueue at the core of it all so I understand the concern. Glad to hear suggestions to make it more acceptable, perhaps smoothing out access to the connection object or protecting against direct access to it etc.