Question about running ethereum's macro benchmark using Smallbank workload

[nicodechal]

Question about smallbank driver

I'm trying to run benchmark using Smallbank workload,But I'm not sure which one is right. I saw the comment below: https://github.com/ooibc88/blockbench/blob/d722cab5239dbd9938f5a99ab4f5aa8b1e7a078e/benchmark/ethereum/start-clients.sh#L18-L19 I'm not clear about what's the meaning of this comment, because from document, it seems that YCSB and Smallbank use different drivers, but code says I only need to add -wl smallbank to test using smallbank workload.

And I tried to use the driver in src/macro/smallbank as document say, but I got a all zero results,like

In the last 2s, tx count = 0 latency = 0 outstanding request = 0
In the last 2s, tx count = 0 latency = 0 outstanding request = 395
In the last 2s, tx count = 0 latency = 0 outstanding request = 783
In the last 2s, tx count = 0 latency = 0 outstanding request = 1169
...

and tx count and latency are always 0, only the outstanding request increase. I'm still not sure about why I got this results

---

Question about code

Another question is about the true in the code below: https://github.com/ooibc88/blockbench/blob/d722cab5239dbd9938f5a99ab4f5aa8b1e7a078e/src/macro/kvstore/ycsbc.cc#L146-L147 If I change this true to false, does this means use YCSB simulate smallbank? because this is document say:

For Ethereum, smallbank is simulated via ycsb

In this way I can get some results, but the throughput I calculate is very large, like 10 times larger than YCSB workload. Is this result reasonable?

Hope to get help!

[yolo-612]

Hello, I‘m sorry that I’m not answering the question but asking the question. I'm just starting the simulation of the ethereum macro benchmark. It is totally new for me .. from where to start .. dont know .. Is there any manual (step by step) ? can you help me

[ug93tad]

The comment in "start-client.sh" means that both the executable for YCSB and Smallbank are named "driver". You need to change the path to the correct workload.

The "true" option in ycsbc.cc is to pre-load the data, has nothing to do with Smallbank.

"smallbank is simulated via ycsb" means that it simply carries multiple updates of different keys instead of checking for account/balance/etc. The smallbank driver for Hyperledger is a more precise implementation.

[nicodechal]

@ug93tad thank you for your reply!

So in Q2, when i change the true to false, then what's the meaning of the code below(Line 51, it uses method called DoTransaction)?

https://github.com/ooibc88/blockbench/blob/d722cab5239dbd9938f5a99ab4f5aa8b1e7a078e/src/macro/kvstore/ycsbc.cc#L47-L52

---

Also, I have tried to use Smallbank workload in src/macro/smallbank, but I got some results like I mentioned before, Is there any possible reason for this result?

[ug93tad]

For Ethereum, both YCSB and Smallbank use the driver in ../macro/kvstore (see the Readme). All operations are "DoInsert()". The "DoTransaction()" is relics from the YCSB code we ported (https://github.com/basicthinker/YCSB-C).

The results you showed indicate that the server didn't produce any blocks. Please check your server logs to see what went wrong.

[nicodechal]

@ug93tad So if I want to test Ethereum using smallbank, I dont need to change driver and only need to use -wl smallbank option?

[ug93tad]

No, sorry my mistake. Smallbank uses the driver in ../smallbank directory

[nicodechal]

@ug93tad Thanks. So can I use -wl smallbank with YCSB workload to simulate smallbank workload? because the result using driver in ../smallbank still give me wrong results and no error in server logs ... And I saw the usage in code: https://github.com/ooibc88/blockbench/blob/d722cab5239dbd9938f5a99ab4f5aa8b1e7a078e/src/macro/kvstore/ycsbc.cc#L253-L254 Can this still work?

[ug93tad]

yes, but for simulated workload only, and only for Ethereum. The correct smallbank should be run using the driver in ../smallbank folder.