skyscraper
skyscraper copied to clipboard
muldjord
World Of Spectrum not scraping
World of Spectrum appears to no longer be scraping. FIles are named as pretty much just the game name.
It looks like the style of the site has changed, so i guess this would need adaptation.
There is also a newer ZX database here: https://spectrumcomputing.co.uk/ that was intended by many to replace WoS ... might be worth considering?
Do you know the people behind spectrumcomputing.co.uk? I need permission to scrape from them.
WoS now has an API it seems. It requires a key. If you can provide me with one, I can update the implementation.
This is how you use the new API, and you can just use 'test' as the API key. you need to follow redirects (301) as well (hence the -L option)
curl -s -L "http://worldofspectrum.org/infoseek/api/software?X-API-KEY=test&title=yumiko+in+the+haunted+mansion"
Returns a JSON like
{"count":1,"totalRecords":"1","titles":[{"id":24171,"title":"Yumiko in the Haunted Mansion","slug":"yumiko-in-the-haunted-mansion-fun-forge","turn_types_text":null,"turn_type_id":null,"availability_id":"1","availability_text":"Available","no_players":"1","entry_type_id":"11","entry_type_text":"Arcade: Maze","entry_group":"games","is_x_rated":"0","is_crap":"0","comments":null,"roles":[{"role":"Music","role_slug":"music","person":"Yerzmyey","person_slug":"yerzmyey","person_id":"6163","role_id":"22241"},{"role":"(Unknown)","role_slug":"unknown","person":"Leszek Chmielewski Daniel","person_slug":"leszek-chmielewski-daniel","person_id":"1460","role_id":"22242"},{"role":"(Unknown)","role_slug":"unknown","person":"Yerzmyey","person_slug":"yerzmyey","person_id":"6163","role_id":"22243"}],"features":[{"id":"37","feature":"TR-DOS Support","slug":"tr-dos-support"}],"controls":[{"id":"2","control":"Interface 2 (right)","slug":"interface-2-right"},{"id":"4","control":"Kempston","slug":"kempston"},{"id":"5","control":"Redefinable Keys","slug":"redefineable-keys"}],"themes":[],"series":[],"publishers":[{"id":"4604","name":"Fun Forge","slug":"Fun-Forge"}]}]}
Also There is a DB at zxdb/ZXDB which I believe is tied to https://api.zxinfo.dk/v3/ It seems to be a db combining all of these zx spectrum sites:
- http://www.worldofspectrum.org/
- http://www.zxspectrumreviews.co.uk/
- http://www.rzxarchive.co.uk/
- http://www.users.globalnet.co.uk/~jg27paw4/spot-on/
I plan on updating my fork of skyscraper to use the new WoS api and hopefully incorporate this as well.
I started working on implementing the new API when you wrote the last message, but if you end up doing so, a PR would be appreciated for that feature. Thanks. I doubt I will complete my implementation anytime soon.