apache
Test Performance Failure
Hi developer, I'm having troubles in testing PBFT performance.
I launched 5 AWS instances as nodes for resdb. While running the command:
./performance/pbft_performance.sh config/kv_performance_server.conf
to test performance, I encountered the following issue:
During the execution of the script, the client node suddenly disconnects, resulting in an incorrect number of nodes being counted when calculating the results.
I checked the AWS console for the affected node’s resource usage (the instance type is c5a.xlarge), and here are the observations:
- CPU usage peaked at 37%.
- Network traffic reached a maximum of 110M/220M (in/out).
Hi Shubo, can you provide the detailed logs for the replica and client (defaultly in replica's /root/kv_server_performance.log) ?
Hi Shubo,
I saw you used c5a.xlarge which only contains 8GB mem. The memory is not enough as the performance tool suit will generate a large amount of requests.
You can reduce the number of requests by changing the number here: https://github.com/apache/incubator-resilientdb/blob/master/platform/consensus/ordering/pbft/performance_manager.cpp#L111 Like 60000000 to 2000000.
Hope this can help you.
Thanks, Junchao
On Fri, Jan 17, 2025 at 9:18 PM Shubo Peng @.***> wrote:
Hi developer, I'm having troubles in testing PBFT performance.
I launched 5 AWS instances as nodes for resdb. While running the command:
./performance/pbft_performance.sh config/kv_performance_server.conf
to test performance, I encountered the following issue:
2025-01-17.21-13-27.png (view on web) https://github.com/user-attachments/assets/0e04601e-5d71-41e7-8d13-9ff0c91f6a1b
During the execution of the script, the client node suddenly disconnects, resulting in an incorrect number of nodes being counted when calculating the results.
2025-01-17.21-14-31.png (view on web) https://github.com/user-attachments/assets/b02b50de-1406-4012-8b54-0bd409a10045
I checked the AWS console for the affected node’s resource usage (the instance type is c5a.xlarge), and here are the observations:
- CPU usage peaked at 37%.
- Network traffic reached a maximum of 110M/220M (in/out).
— Reply to this email directly, view it on GitHub https://github.com/apache/incubator-resilientdb/issues/169, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4YYLTIRKOCAIXWEHRNB5L2LD7MFAVCNFSM6AAAAABVL5CM46VHI2DSMVQWIX3LMV43ASLTON2WKOZSG44TKNBWGYYDMNY . You are receiving this because you are subscribed to this thread.Message ID: @.***>
Hi Junchao,
I noticed that resdb was able to support smart contracts after the Feb 14th update, so I wanted to change the KVRequest to the contract's request when measuring pbft performance. specifically, I changed KVRequest::SET in benchmark/protocols/pbft/kv_server_performance.cpp to Request::CREATE_ACCOUNT (I saw this usage in executor/contract/executor/contract_executor_test.cpp), but the problem is that I can't find the Request::CREATE_ACCOUNT is defined somewhere.
Another question. I tried to control the send rate of request (like 30,000 reqs/sec) when testing, so I added some code here:
https://github.com/apache/incubator-resilientdb/blob/master/platform/consensus/ordering/pbft/performance_manager.cpp#L111
But it didn't work. It ran out the same as if I hadn't changed it.
Hi Shubo,
Thanks for your interest in the smart contract.
The interface is different from the original Key-Value Service, the request is a sub-message inside the KvRequest. You can take a look here how to generate the request on Create-Account for smart contracts.
https://github.com/apache/incubator-resilientdb/blob/master/interface/kv/contract_client.cpp#L35
I have done the performance on the smart contracts. Let me know if you have any issues while running the performance.
Thanks, Junchao
On Mon, Feb 17, 2025 at 6:36 PM Shubo Peng @.***> wrote:
Hi Junchao,
I noticed that resdb was able to support smart contracts after the Feb 14th update, so I wanted to change the KVRequest to the contract's request when measuring pbft performance. specifically, I changed KVRequest::SET in benchmark/protocols/pbft/kv_server_performance.cpp to Request::CREATE_ACCOUNT (I saw this usage in executor/contract/executor/contract_executor_test.cpp), but the problem is that I can't find the Request::CREATE_ACCOUNT is defined somewhere.
Another question. I tried to control the send rate of request (like 30,000 reqs/sec) when testing, so I added some code here:
https://github.com/apache/incubator-resilientdb/blob/master/platform/consensus/ordering/pbft/performance_manager.cpp#L111 http://url 2025-02-17.183009.png (view on web) https://github.com/user-attachments/assets/6f560f95-b5a8-4848-beca-38f2a39a7f22 But it didn't work. It ran out the same as if I hadn't changed it.
— Reply to this email directly, view it on GitHub https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2662702512, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4YYLW3NYLYRKUVFVMEA2T2QG3RLAVCNFSM6AAAAABVL5CM46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRSG4YDENJRGI . You are receiving this because you commented.Message ID: @.***> [image: Shubo-Peng]Shubo-Peng left a comment (apache/incubator-resilientdb#169) https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2662702512
Hi Junchao,
I noticed that resdb was able to support smart contracts after the Feb 14th update, so I wanted to change the KVRequest to the contract's request when measuring pbft performance. specifically, I changed KVRequest::SET in benchmark/protocols/pbft/kv_server_performance.cpp to Request::CREATE_ACCOUNT (I saw this usage in executor/contract/executor/contract_executor_test.cpp), but the problem is that I can't find the Request::CREATE_ACCOUNT is defined somewhere.
Another question. I tried to control the send rate of request (like 30,000 reqs/sec) when testing, so I added some code here:
https://github.com/apache/incubator-resilientdb/blob/master/platform/consensus/ordering/pbft/performance_manager.cpp#L111 http://url 2025-02-17.183009.png (view on web) https://github.com/user-attachments/assets/6f560f95-b5a8-4848-beca-38f2a39a7f22 But it didn't work. It ran out the same as if I hadn't changed it.
— Reply to this email directly, view it on GitHub https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2662702512, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4YYLW3NYLYRKUVFVMEA2T2QG3RLAVCNFSM6AAAAABVL5CM46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRSG4YDENJRGI . You are receiving this because you commented.Message ID: @.***>
Hi Junchao,
I've included interface/kv/contract_client.h when I changed KVRequest to Request, but it didn't work.
Do I need to include other libraries?
Thanks, Shubo
Hi Shubo,
What error did you get? If it did not find "Request::CREATE_ACCOUNT", could you try to use resdb::contract::Request::CREATE_ACCOUNT?
Thanks, Junchao
On Tue, Feb 18, 2025 at 3:16 PM Shubo Peng @.***> wrote:
Hi Junchao,
I've included interface/kv/contract_client.h when I changed KVRequest to Request, but it didn't work.
Do I need to include other libraries?
Thanks, Shubo
— Reply to this email directly, view it on GitHub https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2664795946, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4YYLW3O5THSOH3IFPZQCD2QLM3NAVCNFSM6AAAAABVL5CM46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRUG44TKOJUGY . You are receiving this because you commented.Message ID: @.***> [image: Shubo-Peng]Shubo-Peng left a comment (apache/incubator-resilientdb#169) https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2664795946
Hi Junchao,
I've included interface/kv/contract_client.h when I changed KVRequest to Request, but it didn't work.
Do I need to include other libraries?
Thanks, Shubo
— Reply to this email directly, view it on GitHub https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2664795946, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4YYLW3O5THSOH3IFPZQCD2QLM3NAVCNFSM6AAAAABVL5CM46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRUG44TKOJUGY . You are receiving this because you commented.Message ID: @.***>
Hi Junchao,
Thank you for your reply. That's the problem, I found out later to use resdb::contract::Request::CREATE_ACCOUNT.
And can you help me with my another problem mentioned above?
Thanks, Shubo
Hi Shubo,
The source code you mentioned is to generate the data: https://github.com/apache/incubator-resilientdb/blob/master/platform/consensus/ordering/pbft/performance_manager.cpp#L111 http://url/
You can set some the sleep here: https://github.com/apache/incubator-resilientdb/blob/master/platform/consensus/ordering/pbft/performance_manager.cpp#L294
Thanks, Junchao
On Tue, Feb 18, 2025 at 10:16 PM Shubo Peng @.***> wrote:
Hi Junchao,
Thank you for your reply. That's the problem, I found out later to use resdb::contract::Request::CREATE_ACCOUNT.
And can you help me with my another problem mentioned above?
Thanks, Shubo
— Reply to this email directly, view it on GitHub https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2665843978, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4YYLVJYCEFNMDVOFNJYT32QM6DDAVCNFSM6AAAAABVL5CM46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRVHA2DGOJXHA . You are receiving this because you commented.Message ID: @.***> [image: Shubo-Peng]Shubo-Peng left a comment (apache/incubator-resilientdb#169) https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2665843978
Hi Junchao,
Thank you for your reply. That's the problem, I found out later to use resdb::contract::Request::CREATE_ACCOUNT.
And can you help me with my another problem mentioned above?
Thanks, Shubo
— Reply to this email directly, view it on GitHub https://github.com/apache/incubator-resilientdb/issues/169#issuecomment-2665843978, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4YYLVJYCEFNMDVOFNJYT32QM6DDAVCNFSM6AAAAABVL5CM46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRVHA2DGOJXHA . You are receiving this because you commented.Message ID: @.***>