tranco-list icon indicating copy to clipboard operation
tranco-list copied to clipboard
verified publisher icon DistriNet

Issues Downloading Full List

Open peterstory opened this issue 3 years ago • 1 comments

Hello! I'm trying to download the full list for June 29th, 2022: https://tranco-list.eu/list/Q9X24/1000000

I have no trouble downloading the top 1 million, but when I try to download the full list, I get a network error after 198MB have been downloaded. I've tried at least five times, always with the same result. Since I'm on-campus and have excellent internet speed, I don't think the issue is my connection. Perhaps there's some kind of server-side configuration error that is ending long-lived connections, thereby preventing the download of large files?

peterstory avatar Oct 13 '22 13:10 peterstory

We've been running into the same issue and have been trying on and off for the last few weeks. Cloudflare just returns a 502 bad gateway.

@VictorLeP can we discuss how to fix this? Tranco appears to be the best available post-Alexa option now. Maybe a direct S3 file transfer to save on transfer costs? Or maybe start charging for the full dump?

mfrye avatar Oct 27 '22 17:10 mfrye

We've changed the backend infrastructure, so full list downloads should be stable (they have been for a while now).

VictorLeP avatar Feb 09 '24 13:02 VictorLeP