Report: Benchmark of UUID Feature / Suggest: Remove UUID Feature Or Renew It

[loloicci]

Report: Benchmark of UUID Feature

github workflow: https://github.com/loloicci/line-cosmwasm/actions/runs/8520947517/job/23438536428#step:6:276

Result

• As the result of improving the message and making it short, WASM native can generate a UUID as fast as using API. (takes 102% time than original)
• But, using this message, API is more faster (takes 84 % time than WASM native)
• Our defined gas cost for using API is not enough good. In some cases (e.g. uuid_api_concat), it is much cheaper. (uuid_api_separate vs uuid_api_concat, takes 0.99% time and costs 82% gas)

Description of the benchmark

Benchmarking uuid/without_uuid

This does other than generating a UUID. This is for comparison only making UUID part in later benchmarking.

Contract

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/contracts/bench-uuid/src/contract.rs#L49-L54

Result

uuid/without_uuid       time:   [984.65 ns 989.35 ns 993.75 ns]

Gas used for dummy: 50,550,000

Benchmarking uuid/uuid_original

• This is the original method in the main branch. It uses Api/sha1_calculate.
• This uses [ADDRESS] [HEIGHT_IN_DECIMAL] [SEQ_IN_DECIMAL] as the message.

Contract

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/contracts/bench-uuid/src/contract.rs#L56-L61

UUID Generation

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/packages/std/src/uuid.rs#L79

Result

uuid/uuid_original      time:   [6.9735 µs 6.9961 µs 7.0331 µs]

Gas used for original: 1,475,250,047

Benchmarking uuid/uuid_api

• This uses a new method that takes multiple messages to generate UUID and uses Api/sha1_calculate.
• This uses [ADDRESS] [HEIGHT_IN_DECIMAL] [SEQ_IN_DECIMAL] as the message.

Contract

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/contracts/bench-uuid/src/contract.rs#L63-L68

UUID Generation

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/packages/std/src/uuid.rs#L98

Result

uuid/uuid_api           time:   [7.2346 µs 7.2399 µs 7.2462 µs]

Gas used for api: 1,554,600,046

Benchmarking uuid/uuid_api_separate

• This uses a new method that takes multiple messages to generate UUID and uses Api/sha1_calculate.
• This uses improved style message [ADDRESS][HEIGHT_AS_U8x8][SEQ_AS_U8x8].
• The message is given separated as [ADDRESS], [HEIGHT_AS_U8x8], and [SEQ_AS_U8x8].

Contract

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/contracts/bench-uuid/src/contract.rs#L70-L75

UUID Generation

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/packages/std/src/uuid.rs#L117

Result

uuid/uuid_api_separate  time:   [6.2340 µs 6.2369 µs 6.2400 µs]

Gas used for api separate: 1,557,050,047

Benchmarking uuid/uuid_api_concat

• This uses the same method as uuid_original to generate a UUID and uses Api/sha1_calculate.
• This uses improved style message [ADDRESS][HEIGHT_AS_U8x8][SEQ_AS_U8x8].
• The message is given after concated.

Contract

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/contracts/bench-uuid/src/contract.rs#L77-L82

UUID Generation

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/packages/std/src/uuid.rs#L135

Result

uuid/uuid_api_concat    time:   [6.1737 µs 6.1778 µs 6.1814 µs]

Gas used for api concat: 1,272,450,047

Benchmarking uuid/uuid_wasm

• This uses a new method that takes multiple messages to generate UUID and does not use Api but uses only WASM native calculation.
• This uses improved style message [ADDRESS][HEIGHT_AS_U8x8][SEQ_AS_U8x8].
• The message is given separated as [ADDRESS], [HEIGHT_AS_U8x8], and [SEQ_AS_U8x8].

Contract

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/contracts/bench-uuid/src/contract.rs#L84-L89

UUID Generation

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/packages/std/src/uuid.rs#L152

Result

uuid/uuid_wasm          time:   [7.1432 µs 7.1905 µs 7.2736 µs]

Gas used for wasm: 1,566,600,047

Benchmarking uuid/uuid_wasm_concat

• This uses a new method that takes multiple messages to generate UUID and does not use Api but uses only WASM native calculation.
• This uses the same method as uuid_original to generate a UUID and uses Api/sha1_calculate.
• This uses improved style message [ADDRESS][HEIGHT_AS_U8x8][SEQ_AS_U8x8].
• The message is given after concated.

Contract

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/contracts/bench-uuid/src/contract.rs#L91-L96

UUID Generation

https://github.com/loloicci/line-cosmwasm/blob/d3d71312397ec19927e633902db89ece592ef310/packages/std/src/uuid.rs#L168

Result

uuid/uuid_wasm_concat   time:   [7.3775 µs 7.3815 µs 7.3857 µs]

Gas used for wasm concat: 1,628,400,047

Suggestion

There are two different suggestions. We need to discuss which is better.

Remove Sha1 API

Removing Sha1 API from our cosmwasm and using the new style messages for generating UUIDs.

Using new style messages, it takes almost the same time to generate a UUID as before even if we do not use API.

Improve the Message And Modify the Gas Cost

Improving the message style and where the messages are concatenated.

These make generating UUIDs faster. But, If we adopt it, we need to improve how to calculate the gas cost of the API. Because this benchmark tells us this makes the gas cost cheaper than the proper cost.

[170210]

@loloicci Thank you, but could you explain more about

But, If we adopt it, we need to improve how to calculate the gas cost of the API. Because this benchmark tells us this makes the gas cost cheaper than the proper cost.

It seems uuid/uuid_api_concat is the fastest method, and the gas fee is the lowest.

[da1suk8]

If the time it takes to generate is not much different, then I think it's fine to consider removing it.

Thank you.

[loloicci]

It seems uuid/uuid_api_concat is the fastest method, and the gas fee is the lowest.

The gas cost for calling API is defined by us. It should be in direct proportion to the taken time. Gas cost for uuid/uuid_api_concat is cheaper than proper in comparison to others. (and uuid/uuid_api_separate takes much expensive gas)