[RFC] mnesia: Add post-commit hook

[ieQu1]

Hello, OTP team.

Our team has been working on improving horizontal scalability of mnesia by making replication partially eventually consistent: "whitepaper", source code. The results are quite promising, we've been able to run 20+ node mria/mnesia clusters with high write and read throughput. However, in order to do this we had to add a hook point in mnesia_tm. Although the patch is really small, ideally we would like to merge it upstream.

Currently this PR is WIP, it lacks at least two things:

• Documentation
• Tests

If you are interested in accepting this patch, I will fix both issues.

Documentation is missing because I am not sure how to present this feature. It exposes some internal data structures, so it's not meant for use in the business logic. So the documentation should either go with a large disclaimer that the backward compatibility is not guaranteed, or, perhaps, it can be omitted altogether. As for tests, we have some in mria repo, but I am not sure in which test suite in OTP can we put it. Some advice is welcome.

Thank you.

[CLAassistant]

All committers have signed the CLA.

[github-actions[bot]]

CT Test Results

Tests are running... https://github.com/erlang/otp/actions/runs/2207693668

Results for commit 743d12e3d43d2d63a03dd03df1b1a7c758c64f7d

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

• No CT logs found
• No HTML docs found
• No Windows Installer found

// Erlang/OTP Github Action Bot

[ieQu1]

Hi,

Sorry I missed that "waiting" label has been applied to the ticket. What kind of input can I provide?

[dgud]

I think the wrong label was applied, sorry for taking some time, this is stuff that goes to the bottom of the priority list for us :-/

And I have forgotten the answer (I'm old and vacation) to why this could not be in commit_work?

[ieQu1]

Hi @dgud ,

No worries whatsoever. The answer is that we want to import an entire commit record (all the ops for all tables) at once, while commit_work is called per op, unless I also forgot some details.

[ieQu1]

P.S. There is another reason where it can be useful: other storage backends (such as rocksdb) can use this API for batch writes.