speedtest_exporter icon indicating copy to clipboard operation
speedtest_exporter copied to clipboard

Accuracy of this speedtest code is unknown

Open virtuallynathan opened this issue 3 years ago • 8 comments

Speedtests can be pretty all over the place, I haven't had a chance to fully evaluate this one, but with a few initial tests it does not appear to be very accurate. I think for maximum accuracy, just exec'ing the official speedtest.net binary would probably be best. It will output json and other formats.

Great work overall on this project, I just set it up.

virtuallynathan avatar May 15 '21 21:05 virtuallynathan

Target Server: [35180] 17.14km Seattle, WA (United States) by Ziply Fiber Latency: 2.483086ms Download Test: . Upload Test: .

Download: 733.27 Mbit/s Upload: 866.26 Mbit/s

vs:

Server: Ziply Fiber - Seattle, WA (id = 35180)
    ISP: Ziply Fiber
Latency:     1.95 ms   (0.16 ms jitter)

Download: 919.35 Mbps (data used: 450.2 MB) Upload: 934.45 Mbps (data used: 420.9 MB)

virtuallynathan avatar May 15 '21 21:05 virtuallynathan

Totally agree, every speed test shows different results, but by using the same one yields some kind of consistency. For most users (me included at the point of writing it) it only needed to show tends. For me it was to tend the speed changes as Starlink release software changes to their hardware.

I have not looked deeply into it, but sometimes it feels Docker is constraining the speedtest, but then I'm running it on Mac so Docker is inside a VM aswell.

I did POC using Speedtests.net CLI binary but it started to get messy (cant remember exactly but I ended up with external dependencies) then a bigger final image size, not running on distroless/static, so included vulnerabilities. and needing to accept end user agreements.

I may try again now I have my external Starlink monitoring up and running, I can iterate on the exporters and work on accuracy.

😉 If only Netflix would release more information around fast.com

dwillcocks avatar May 16 '21 11:05 dwillcocks

That makes sense, I may take a look at writing some code to run the speedtest-cli and see how messy it becomes. I think i'll also poke around with the Go speedtest impl you are using and see if any tweaks make it more accurate.

If only... haha.

virtuallynathan avatar May 16 '21 12:05 virtuallynathan

If only... haha.

fast_exporter maybe!?!

dwillcocks avatar May 16 '21 13:05 dwillcocks

I agree with accuracy problem. I am currently getting negative results. These make it unusable.

curl http://localhost:9092/metrics
# HELP speedtest_download_speed_Bps Last download speedtest result
# TYPE speedtest_download_speed_Bps gauge
speedtest_download_speed_Bps{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} 7.87970036405554e+07
# HELP speedtest_latency_seconds Measured latency on last speed test
# TYPE speedtest_latency_seconds gauge
speedtest_latency_seconds{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} 2.058390779
# HELP speedtest_scrape_duration_seconds Time to preform last speed test
# TYPE speedtest_scrape_duration_seconds gauge
speedtest_scrape_duration_seconds{test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb"} 25.850604899
# HELP speedtest_up Was the last speedtest successful.
# TYPE speedtest_up gauge
speedtest_up{test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb"} 1
# HELP speedtest_upload_speed_Bps Last upload speedtest result
# TYPE speedtest_upload_speed_Bps gauge
speedtest_upload_speed_Bps{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} -2.2512346881061174e+07

CharlesGodwin avatar Aug 10 '21 12:08 CharlesGodwin

Is there any way to encourage you to deal with this. I am currently getting persistent negative values for uploads. I'd look inot it myself bu I have no experience with Prometheus or Go.

Thank you

CharlesGodwin avatar Aug 11 '21 12:08 CharlesGodwin

Same for me, unfortunately, the results are inaccurate. Any we can do to help ?

keyaertc avatar Mar 08 '22 08:03 keyaertc

Same for me, constant negative upload speeds

jbartak avatar Aug 06 '22 17:08 jbartak