ic icon indicating copy to clipboard operation
ic copied to clipboard

perf(consensus): avoid heavy `IngressMessageId` computation ingress during payload creation in ingress selector

Open kpop-dfinity opened this issue 2 months ago • 0 comments

Instead of computing IngressMessageId (which requires computing a hash of the whole message) twice for each ingress message included in an ingress payload, we now use the precomputed id stored as part of IngressPoolObj.

Note that for small payload sizes the payload creation is fast already, but for larger payloads the gain might be noticeable.

Benchmark results

get_ingress_payload/payload size limit: 4194304, canisters: 100000, ingress pool size: 100000, ingre...
                        time:   [61.380 ms 62.040 ms 62.743 ms]
                        change: [-24.094% -22.921% -21.761%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 4194304, canisters: 1, ingress pool size: 100000, ingress me...
                        time:   [27.606 ms 28.342 ms 29.080 ms]
                        change: [-33.645% -31.687% -29.901%] (p = 0.00 < 0.05)
                        Performance has improved.
                        
get_ingress_payload/payload size limit: 4194304, canisters: 1000, ingress pool size: 1000, ingress m...
                        time:   [8.4004 ms 8.4775 ms 8.5583 ms]
                        change: [-59.120% -58.687% -58.202%] (p = 0.00 < 0.05)
                        Performance has improved.
                        
get_ingress_payload/payload size limit: 4194304, canisters: 1, ingress pool size: 1000, ingress mess...
                        time:   [7.4612 ms 7.5229 ms 7.5984 ms]
                        change: [-61.734% -61.361% -60.942%] (p = 0.00 < 0.05)
                        Performance has improved.
                        
get_ingress_payload/payload size limit: 4194304, canisters: 1, ingress pool size: 2, ingress message...
                        time:   [3.1129 ms 3.1311 ms 3.1494 ms]
                        change: [-64.660% -64.483% -64.284%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 4194304, canisters: 1, ingress pool size: 100, ingress messa...
                        time:   [4.6602 ms 4.6782 ms 4.6972 ms]
                        change: [-61.146% -60.943% -60.748%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 8388608, canisters: 100000, ingress pool size: 100000, ingre...
                        time:   [62.216 ms 62.762 ms 63.380 ms]
                        change: [-29.718% -28.717% -27.721%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 8388608, canisters: 1, ingress pool size: 100000, ingress me...
                        time:   [26.879 ms 27.025 ms 27.188 ms]
                        change: [-50.733% -50.221% -49.691%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 8388608, canisters: 1000, ingress pool size: 1000, ingress m...
                        time:   [12.222 ms 12.262 ms 12.303 ms]
                        change: [-60.827% -60.628% -60.440%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 8388608, canisters: 1, ingress pool size: 1000, ingress mess...
                        time:   [11.121 ms 11.164 ms 11.209 ms]
                        change: [-62.948% -62.756% -62.553%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 8388608, canisters: 1, ingress pool size: 4, ingress message...
                        time:   [7.1523 ms 7.1814 ms 7.2117 ms]
                        change: [-62.234% -62.057% -61.895%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 8388608, canisters: 1, ingress pool size: 100, ingress messa...
                        time:   [9.4688 ms 9.5372 ms 9.6033 ms]
                        change: [-57.324% -57.011% -56.729%] (p = 0.00 < 0.05)
                        Performance has improved.
                        
get_ingress_payload/payload size limit: 16777216, canisters: 100000, ingress pool size: 100000, ingr...
                        time:   [102.59 ms 103.52 ms 104.63 ms]
                        change: [-17.965% -17.007% -15.907%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 16777216, canisters: 1, ingress pool size: 100000, ingress m...
                        time:   [40.903 ms 41.587 ms 42.223 ms]
                        change: [-42.027% -40.960% -40.068%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 16777216, canisters: 1000, ingress pool size: 1000, ingress ...
                        time:   [21.238 ms 21.516 ms 21.794 ms]
                        change: [-58.133% -57.642% -57.106%] (p = 0.00 < 0.05)
                        Performance has improved.
                        
get_ingress_payload/payload size limit: 16777216, canisters: 1, ingress pool size: 1000, ingress mes...
                        time:   [18.793 ms 18.846 ms 18.902 ms]
                        change: [-63.587% -63.388% -63.188%] (p = 0.00 < 0.05)
                        Performance has improved.
                        
get_ingress_payload/payload size limit: 16777216, canisters: 1, ingress pool size: 8, ingress messag...
                        time:   [14.592 ms 14.690 ms 14.798 ms]
                        change: [-62.303% -62.058% -61.757%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 16777216, canisters: 1, ingress pool size: 100, ingress mess...
                        time:   [16.084 ms 16.139 ms 16.194 ms]
                        change: [-61.339% -61.186% -61.020%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 33554432, canisters: 100000, ingress pool size: 100000, ingr...
                        time:   [105.95 ms 106.82 ms 107.68 ms]
                        change: [-38.193% -37.598% -37.019%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 33554432, canisters: 1, ingress pool size: 100000, ingress m...
                        time:   [47.280 ms 47.440 ms 47.616 ms]
                        change: [-55.197% -54.922% -54.667%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 33554432, canisters: 1000, ingress pool size: 1000, ingress ...
                        time:   [34.532 ms 34.640 ms 34.751 ms]
                        change: [-61.665% -61.496% -61.323%] (p = 0.00 < 0.05)
                        Performance has improved.
                        
get_ingress_payload/payload size limit: 33554432, canisters: 1, ingress pool size: 1000, ingress mes...
                        time:   [33.501 ms 33.696 ms 33.922 ms]
                        change: [-62.099% -61.799% -61.493%] (p = 0.00 < 0.05)
                        Performance has improved.
                        
get_ingress_payload/payload size limit: 33554432, canisters: 1, ingress pool size: 16, ingress messa...
                        time:   [28.702 ms 28.757 ms 28.814 ms]
                        change: [-63.996% -63.847% -63.691%] (p = 0.00 < 0.05)
                        Performance has improved.

get_ingress_payload/payload size limit: 33554432, canisters: 1, ingress pool size: 100, ingress mess...
                        time:   [30.592 ms 30.643 ms 30.696 ms]
                        change: [-61.991% -61.821% -61.656%] (p = 0.00 < 0.05)
                        Performance has improved.

kpop-dfinity avatar Nov 17 '25 09:11 kpop-dfinity