Introduce mutexes to lock access to logs

[adzialocha]

.. for example for incrementing the log id

[cafca]

I know you wrote something about this before, do you have a link to that or can you copy it here? I forgot what the idea was exactly (because the db also has locks).

[adzialocha]

I know you wrote something about this before, do you have a link to that or can you copy it here?

Can't remember where I've written something, maybe in the chat? :cry:

I forgot what the idea was exactly

We have a couple of potential race conditions in our validation flow where we have async checks before things land in the database:

check_this(entry).await?;
check_that(entry).await?;
insert_db(entry).await?;

This could potentially cause race conditions if we imagine requests to run concurrently in multiple tasks for the same log_id. For example there might be a condition where you fastly send two entries (sequence numbers 1 and then 2) of the same log after each other, but the validation for the second entry kicks in earlier than the insertion of the first, the second request will be rejected now even though it technically was not "invalid" and sent in the correct order by the client.

(because the db also has locks).

Yes, that could be solved with a database! We should definitely explore that option and I think it could work. Though I'm not 100% sure yet if thats a good architectual decision (need to think about it more) as database are not the only sources of asynchronicity (networking, mutexes and potentially other things in the future ..) and it might be good to have a system "on top" of all that, independent of something more specialised like a database. I find it unintuitive to use databases to also solve locking for a networking request, an independent thin "Lock" service can be tested isolated and is explicit, instead of being hidden in another layer which is mainly about persistence.

I think locking is great within multiple database transactions, thats cool and we should have that on top of that :+1:

[adzialocha]

We're getting rid of logs 😎