etix
Creating customised JSON download page
I plan on evaluating mirrorbits over the next week or so before we potentially swap out mirrorbrain. One of the key features that attracts us to mirrorbits is the customized download page feature:
"mirrorbits also has a JSON query API to be able to generate your own customized download page for showing alternate mirrors or a sponsor logo along with a countdown"
This is something we had planned on trying to put together on mirrorbrain for sometime but never got round to it. Is there any documentation / sample pages on how to do this? I had a quick look in the repo but couldn't see anything obvious....
thanks
Hi prae5,
First you need to know that it requires a bit of coding though but it's quite easy. The best example of achieving this is certainly the VLC download page.
The first step is to configure mirrorbits in API mode by switching the OutputMode to "API" in the configuration. You can then query any file served by mirrorbits to receive a JSON output instead of a redirect.
For example, querying http://my.server:8080/vlc/2.1.5/win32/vlc-2.1.5-win32.exe will return something similar to:
{
"ClientInfo": {
"ASName": "Comcast Cable Communications, Inc.",
"ASNetmask": 11,
"ASNum": 7922,
"AreaCode": 650,
"CharSet": 1,
"City": "San Mateo",
"ContinentCode": "NA",
"CountryCode": "US",
"CountryCode3": "USA",
"CountryName": "United States",
"Latitude": 37.555,
"Longitude": -122.2687,
"PostalCode": "94404",
"Region": "CA"
},
"ExcludedList": [
{
"ASOnly": false,
"AdminEmail": "",
"AdminName": "",
"Asnum": 3323,
"Comment": "",
"ComputedScore": 0,
"ContinentCode": "EU",
"ContinentOnly": false,
"CountryCodes": "GR TR",
"CountryOnly": true,
"CustomData": "",
"Distance": 10921.207,
"Enabled": true,
"ExcludeReason": "Country only",
"FtpURL": "ftp://ftp.ntua.gr/pub/videolan/",
"HttpURL": "http://ftp.ntua.gr/pub/videolan/",
"ID": "ftp.ntua.gr",
"LastSync": 0,
"Latitude": 37.9833,
"Longitude": 23.7333,
"RsyncURL": "",
"Score": 0,
"SponsorLogoURL": "",
"SponsorName": "National Technical University of Athens",
"SponsorURL": "http://www.ntua.gr",
"StateSince": 1408550395
}
],
"FileInfo": {
"ModTime": "2014-07-26T02:30:38+02:00",
"Path": "/vlc/2.1.5/win32/vlc-2.1.5-win32.exe",
"Sha1": "bc5e2b879c110c7702973fa3c380550ea2856689",
"Size": 24743106
},
"IP": "76.126.252.130",
"MirrorList": [
{
"ASOnly": false,
"AdminEmail": "<redacted>",
"AdminName": "",
"Asnum": 26347,
"Comment": "",
"ComputedScore": 35498,
"ContinentCode": "NA",
"ContinentOnly": false,
"CountryCodes": "US",
"CountryOnly": false,
"CustomData": "",
"Distance": 566.41754,
"Enabled": true,
"FtpURL": "",
"HttpURL": "http://mirror.os6.org/videolan/",
"ID": "mirror.os6.org",
"LastSync": 0,
"Latitude": 33.9269,
"Longitude": -117.8612,
"RsyncURL": "rsync://mirror.os6.org/videolan/",
"Score": 10,
"SponsorLogoURL": "",
"SponsorName": "os6.org",
"SponsorURL": "http://www.os6.org",
"StateSince": 1408519957
},
{
"ASOnly": false,
"AdminEmail": "<redacted>",
"AdminName": "",
"Asnum": 13722,
"Comment": "",
"ComputedScore": 18863,
"ContinentCode": "NA",
"ContinentOnly": false,
"CountryCodes": "US",
"CountryOnly": false,
"CustomData": "",
"Distance": 4127.5557,
"Enabled": true,
"FtpURL": "",
"HttpURL": "http://videolan-nyc.defaultroute.com/",
"ID": "videolan-nyc.defaultroute.com",
"LastSync": 0,
"Latitude": 40.7056,
"Longitude": -73.978,
"RsyncURL": "rsync://vlc-2.defaultroute.net/videolan/",
"Score": 0,
"SponsorLogoURL": "//get.videolan.org/sponsors/defaultroute.png",
"SponsorName": "DefaultRoute (New York)",
"SponsorURL": "http://www.defaultroute.com",
"StateSince": 1409096483
},
{
"ASOnly": false,
"AdminEmail": "<redacted>",
"AdminName": "",
"Asnum": 4181,
"Comment": "",
"ComputedScore": 18866,
"ContinentCode": "NA",
"ContinentOnly": false,
"CountryCodes": "US",
"CountryOnly": false,
"CustomData": "",
"Distance": 4125.0527,
"Enabled": true,
"FtpURL": "ftp://ftp.osuosl.org/pub/videolan/",
"HttpURL": "http://ftp.osuosl.org/pub/videolan/",
"ID": "ftp.osuosl.org",
"LastSync": 0,
"Latitude": 40.7143,
"Longitude": -74.006,
"RsyncURL": "rsync://rsync.osuosl.org/videolan/",
"Score": 0,
"SponsorLogoURL": "//get.videolan.org/sponsors/osl.png",
"SponsorName": "Oregon State University Open Source Lab",
"SponsorURL": "https://osuosl.org/",
"StateSince": 1408519825
}
]
}
As you can see the JSON contains different nodes:
- ClientInfo: contains informations about the client that requested the download
- ExcludedList: a list of excluded mirrors, the reason is specified in "ExcludeReason"
- FileInfo: informations about the requested file like its size, the sha1 checksum, ...
- IP: the ip address of the client
and finally
- MirrorList: containing an ordered list of the more suitable mirrors with their position, sponsorURL, sponsorLogo
In your specific case, the most important nodes within the MirrorList are:
- HttpURL: the base URL where to append the requested path to generate the redirect for the user (HttpURL + REQUEST_URI)
- SponsorName, SponsorURL and SponsorLogoURL if you want to show the sponsor of the mirror on your download page
Basically what you need to do is write a small web app (in PHP, Python, ...) that is invoked for each URL on your domain and who will forward the HTTP request to mirrorbits (don't forget to pass the X-Forwarded-For header containing the IP address of the client) and will parse the JSON to generate the final page.
Thanks for this, I will look at putting something together once I am happy with mirrorbits as a replacement for mirrorbrain. Given work commitments at the moment, this will probably take me a couple of weeks :(
Have you thought about adding as sample/simple php web app similar to what you have on the VLC page to repo? I think it would definitely be something that would attract people considering migrating away from other systems. Its something that was frequently asked for from mb.
thanks
I'm already considering the idea of writing few examples to demonstrate the different usages of the mirrorbits API. I hope to have something ready in the upcoming days.
Did you get a chance to put together a few examples / webapp?
My mirrorbrain server has died, so we need to rebuild it. I am really keen to use this as an opportunity to change how we deliver content (and assuming it works well roll it out to our other sites).
I have a few questions if you could spare some time?
- With mirrorbrain you can easily retrieve the hash of a file by appending .md5/.sha1/.sha256 to the end, do you / do you plan to offer support for this in the future? This isn't a huge deal, I can change my build scripts to generate this and upload it if needed - it was quite a nice feature to have.
- One nice feature of mirrorbrain is quickly being able to pull the status of a file using curl - this is something I've often used when troubleshooting issue, i.e curl -sI http://mirrors.xbmc.org/releases/win32/kodi-14.0-Helix_beta3.exe. I know I could write a quick script to parse the json output - it was just a thought I had whilst testing.
- Is there anyway to use mirrorbits with mod_autoindex like you can with mirrorbrain?
- The key reason for me looking to change to mirrorbits is I want to replicate how you have it setup at VLC, where it shows an intermediary page that shows the countdown timer and mirror sponsors details. I really would appreciate if you could provide some samples on how you have achieved this. If I can get this in place in the next day or so - then this would allow me to put mirrorbits into production.