rust-lightning icon indicating copy to clipboard operation
rust-lightning copied to clipboard

Update RGS stats for August 2024.

Open arik-so opened this issue 1 year ago • 12 comments

This updates the RGS metrics to the latest measurements for V2 gossip.

arik-so avatar Aug 22 '24 16:08 arik-so

I think we might wanna remove the processing times considering the amount of variance they might exhibit depending on the environment.

arik-so avatar Aug 22 '24 16:08 arik-so

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.66%. Comparing base (0d7ae86) to head (c4dea14). Report is 97 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3267      +/-   ##
==========================================
+ Coverage   89.82%   90.66%   +0.84%     
==========================================
  Files         125      126       +1     
  Lines      102830   110644    +7814     
  Branches   102830   110644    +7814     
==========================================
+ Hits        92368   100318    +7950     
+ Misses       7752     7740      -12     
+ Partials     2710     2586     -124     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Aug 22 '24 17:08 codecov[bot]

Yea, I think lets remove the processing times. Otherwise lgtm.

TheBlueMatt avatar Aug 22 '24 19:08 TheBlueMatt

Understanding the processing time is very important. People want to know whether app startup latency is going to be 0.5 seconds or 5 seconds or 50 seconds.

Can we include processing time along with which processor was tested? Or bonus points for a couple of data points so we have a range?

moneyball avatar Aug 22 '24 20:08 moneyball

What is the explanation for the data reduction with the new measurement?

moneyball avatar Aug 22 '24 20:08 moneyball

In that case perhaps we should publish processing times for three-four different environments, say a mobile phone, an Intel chipset laptop, and an ARM chipset computer? I could also test-run it on a Pi.

arik-so avatar Aug 22 '24 23:08 arik-so

Any update here?

TheBlueMatt avatar Sep 06 '24 13:09 TheBlueMatt

Mobile benchmarks were blocked on Swift bindings. I'll be able to run them today.

arik-so avatar Sep 06 '24 19:09 arik-so

What were the results @arik-so ?

moneyball avatar Sep 11 '24 18:09 moneyball

Eh, sadly more complicated than anticipated to just run a couple benchmarks on mobile. Still pending Apple reinstating my lapsed developer account, without which Xcode won't allow me to run anything on a real mobile device.

arik-so avatar Sep 11 '24 18:09 arik-so

I asked @jurvis to run the benchmarks instead, and updated the values accordingly. These new times are significantly faster than the old times because the previous times were measured on an iPhone 11 Pro.

arik-so avatar Sep 11 '24 19:09 arik-so

Cool, should this be un-draft-ed now?

TheBlueMatt avatar Sep 11 '24 20:09 TheBlueMatt

Lgtm!

G8XSU avatar Oct 21 '24 23:10 G8XSU