indy-sdk icon indicating copy to clipboard operation
indy-sdk copied to clipboard

Added implementation of MultiWalletMultiTable postgres plugin strategy

Open JoHenrich opened this issue 4 years ago • 15 comments

We saw that the strategy MultiWalletMultiTable was prepared in the code but not yet implemented.

So decided to do that because we experienced some major peformance issues with the implemented strategies so far.

In addition we added another Strategy: MultiWalletSplitDatabaseMultiTable which cluster the tables in different databases depending on the id of the wallet. That lead to a performance boost.

All unit test ran successfull and we tested the strategy in production aswell but we still consider it as experimentall because further tests are required to ensure that there are no undetected misconceptions.

Signed-off-by: Johannes Henrich [email protected]

JoHenrich avatar Nov 25 '21 10:11 JoHenrich

(ci) test this please

WadeBarnes avatar Nov 25 '21 17:11 WadeBarnes

~Running the example script in the postgres plugin directory gives me an error:~

OK re-built/re-tested and it seems to work now :-S

ianco avatar Nov 25 '21 20:11 ianco

~(I get the same error running ./postgres-getting-started-multi.sh, and I created a script to use the new SingleWalletMultiTable strategy and also gives the same error)~

ianco avatar Nov 25 '21 20:11 ianco

... however the ./postgres-cli-test.sh script is working

ianco avatar Nov 25 '21 20:11 ianco

Several of the rust builds are failing in the Jenkins pipeline. @ianco you have access to have a look.

WadeBarnes avatar Nov 26 '21 16:11 WadeBarnes

Several of the rust builds are failing in the Jenkins pipeline. @ianco you have access to have a look.

It's a rust (cargo) version issue, I posted the info into the rocketchat channel. Not sure what is the fix, looks like we may need to update to the nightly rust version

ianco avatar Dec 07 '21 18:12 ianco

A fix for the build issues has been implemented (https://github.com/hyperledger/indy-sdk/pull/2470). Please rebase this PR on the latest from the main branch so it is no longer blocked by the related build issues.

WadeBarnes avatar Jan 21 '22 16:01 WadeBarnes

Something went wrong with your rebase. This PR now includes changes that already exist in the main branch. Please review and redo your rebase so the PR only contains the changes you've implemented. Thanks

Thanks for the quick look. Rebased it to the commit 2d2140f... as you mention in another PR. Does this work for you now?

JoHenrich avatar Feb 02 '22 09:02 JoHenrich

@JoHenrich, You should rebase on the latest in the main branch. Also a rebase should simply add your changes as a single commit. The commit history is showing, what appears to be, your changes added three times with different commit hashes and a few merge commits; https://github.com/hyperledger/indy-sdk/pull/2452/commits. Would you be able to clean that up?

WadeBarnes avatar Feb 02 '22 15:02 WadeBarnes

@JoHenrich, You should rebase on the latest in the main branch. Also a rebase should simply add your changes as a single commit. The commit history is showing, what appears to be, your changes added three times with different commit hashes and a few merge commits; https://github.com/hyperledger/indy-sdk/pull/2452/commits. Would you be able to clean that up?

I must confess im fairly new to open source contributing and the related git things, so please forgive me. I`ll try to rebase it correctly and clean it up

JoHenrich avatar Feb 02 '22 15:02 JoHenrich

No worries. You'll get the hang of it. Thanks for the contribution.

WadeBarnes avatar Feb 02 '22 15:02 WadeBarnes

No worries. You'll get the hang of it. Thanks for the contribution.

Thanks for your patience :) I hope it`s correct now. Only a single commit. If its still wrong please let me know

JoHenrich avatar Feb 02 '22 16:02 JoHenrich

Code looks good, but unit tests are failing for 2 of the wallet strategies:

WALLET_SCHEME=MultiWalletSingleTable cargo test -- --nocapture --test-threads=1 WALLET_SCHEME=MultiWalletSingleTableSharedPool cargo test -- --nocapture --test-threads=1

... also I recommend updating the tests to support the 2 new wallet schemes, soo:

https://github.com/hyperledger/indy-sdk/blob/master/experimental/plugins/postgres_storage/src/lib.rs#L1758

I`ll have a look into that

JoHenrich avatar Feb 08 '22 12:02 JoHenrich

There is one check failing due to a python version issue (it's failing on other PR's as well), I'll look into this today if I have time ...

ianco avatar Feb 09 '22 16:02 ianco

There is one check failing due to a python version issue (it's failing on other PR's as well), I'll look into this today if I have time ...

Thank you very much :)

JoHenrich avatar Feb 15 '22 07:02 JoHenrich