Panache MongoDB - Optimistic lock control

[diogocarleto]

Hi guys, hope you are doing well.

I was thinking about something I've commented in another ticket some time ago, but in that ticket we are talking about MongoDB migrations. What about we include in MongoDB Panache an optimistic lock feature, just like we have in JPA. Spring and other frameworks already have some similar.

I think it is not so complicated to implement, basically we need an annotation @Version or some similar to identify the field used as "version", and some adjusts in MongoOperations the methods update, and persistOrUpdate.

Today when we call update, in the MongoOperations.update we build the query:

BsonValue id = document.get(ID);
BsonDocument query = new BsonDocument().append(ID, id);

We can change for some like:

BsonValue id = document.get(ID);
BsonValue version = document.get(VERSION);
return new BsonDocument().append(ID, id).append(VERSION, version);

In addition to that we add an OptimisticLockException to inform users trying to update outdated documents.

Do you think it can work?

Best.

[quarkus-bot[bot]]

/cc @FroMage, @evanchooly, @loicmathieu

[diogocarleto]

Guys, I put my idea in a branch where we can compare and evolve or discard.

I created a @Version annotation and a OptimisticLockException, changed some methods in MongoOperator, such as persist, update, persistOrUpdate, and delete to check if the entity has the @Version, if there is no @Version the flow still the same.

I've created a class named VersionHandler, it is a kind of helper to inform us when a entity has the @Version, and if the field using the annotation has value, methods to extract the version value, and also to increment the same.

Let's see a snippet about the persist. There is no secret here, we use VersionHandler to check if the entity is using version and increment version value.

public void persist(Object entity) {
    MongoCollection collection = mongoCollection(entity);
    VersionHandler versionUtil = VersionHandler.of(entity);
    if (versionUtil.containsVersionAnnotation()) {
        versionUtil.adjustVersionValue();
    }
    persist(collection, entity);
}

In update the things starts to get interesting:

public void update(Object entity) {
       VersionHandler versionHandler = VersionHandler.of(entity);
       MongoCollection collection = operations.mongoCollection(entity.getClass());
       BsonDocument query = buildUpdateQuery(entity);

       versionHandler.adjustVersionValue();
       UpdateResult updateResult = collection.replaceOne(query, entity);
       if (versionHandler.containsVersionAnnotation() && updateResult.getModifiedCount() == 0) {
           throwOptimisticLockException(entity);
       }
   }

We use the VersionHandler as in persist, so here you see we have a buildUpdateQuery that basically build a query using only ID, or ID and VERSION, so we try to update, get the UpdateResult, and if there is no documents updated and the entity contains @Version, we throw an OptimistcLockException.

The same idea applies to delete and persistOrUpdate. One point I'm not sure is when we call persistOrUpdate(List/Stream), at this moment I decided to not throw Exceptions if an update fail. And for update(List/Stream) I opted to throw OptimisticLockException.

[diogocarleto]

@loicmathieu do you think it makes sense?

[loicmathieu]

@diogocarleto thanks for the prototype, I'll have a look.

[nicolinux72]

@loicmathieu @diogocarleto

Hi, this change could be very useful, especially when the same collections on MongoDb are used simultaneously by Quarkus and SpringData. Do you plan to release it soon?

[diogocarleto]

morning @nicolinux72, I'm waiting for the considerations of @loicmathieu and guys, I think I'll have to do some adjusts related to the part where I'm using reflection, not sure yet.

I'm using this out of the box in one project, with some adjusts, and it is working pretty good.

Best,

[evanchooly]

Honestly, we're creeping closer and closer to an ODM in quarkus which is less than ideal. I would strongly suggest you consider quarkus-morphia if you need more advanced features like this.

[diogocarleto]

Why Quarkus is not ideal for this @evanchooly? So, shall we remove the MongoDB Panache from Quarkus, and use morphia in place? or even remove the Panache itself from the entire Quarkus platform?

[evanchooly]

It's not that quarkus is not ideal. what's not ideal is building yet another ODM rather than properly integrating with an existing one which is what quarkus-morphia does

[diogocarleto]

I see your point, I already did it. I have only to update the PR with the latest version of Quarkus. We have been using this feature in production for more than 1 year already.

[loicmathieu]

@diogocarleto MongoDB with Panache is a "thin layer" on top of MongoDB, so complex functionalities better stands in an ODM like Morphia. But it's always complex to draw the line between what we want to do inside MongoDB with Panache and what we don't want to ...

Both MongoDB with Panache and Morphia have their place in Quarkus, having options is great for developpers (there is multiple database frameworks for relational database in Quarkus too) and we may add other Panache flavors some day.

It's great to have the same API for relational and non-relational database like Panache do, so switching from Hibernate with Panache in one project to MongoDB with Panache in another one is an easy task.

[bleandro]

@diogocarleto @loicmathieu Hi, guys. How is this goind? This feature would be very important for a project I'm currently working on.

Do you guys still plan on releasing it?

[diogocarleto]

I have plans to open a PR for guys to start reviewing in at maximum 2 weeks. Not sure how long or what changes will be needed

[diogocarleto]

Hello guys, hope you are doing well.

So sorry for my long delay in that.

I've just opened a PR to try to handle that, we have being used a similar approach in our project.

I left the following description in the PR:

" This PR is intent to bring optimist lock control to mongodb-panache, similar what we have in JPA/Hibernate. It is important to notice that this first version doesn't support reactive yet, just imperative repository/entity.

To use that we just need to use the annotation @Version in a entity field, such as:

public class Person { public String name

@Version public Long version; }

The optimistic lock control is supported for the methods persist/persistOrUpdate/update for one single entity or many entities, we can check that in the tests in io.quarkus.it.mongodb.panache.OptimisticLockControlRepositoryIntegrationTests and io.quarkus.it.mongodb.panache.OptimisticLockControlEntityIntegrationTests.

The changes mainly resides in io.quarkus.mongodb.panache.common.runtime.MongoOperations.

Like in jpa/hibernate, every persist/update will increment the version value. If an update fail a io.quarkus.mongodb.panache.common.exception.OptimisticLockException is raised, also similar to jpa/hibernate.

Let me know what do you think and improvements we should do here. "

@loicmathieu can you take a look in this please? Please let me know possible improvements we can make here.

[diogocarleto]

Guys, after a first review of @evanchooly, I've just leverage the moment, and implemented optimistic lock control to reactive mongodb, so now both, reactive and non reactive are supported in this PR.

I'm waiting for the next round of reviews.

Best.

[rdgc]

Hi guys, I really appreciate this feature in the MongoDB Panache. I'm using Quarkus+Panache for more than 2y projects and this feature will help me a lot. I'm waiting for this release soon!!

[rodrigoagbatista]

finally, I worked on a project where this would be very useful