veyepar scraping tool

[codersquid]

The current veyepar tool does not use the latest method Carl provides for grabbing conference data. After discussing with him about this, he demonstrated that he has a url that will pull down a list of json files in the format the pyvideo expects. If we use that method, we won't need the existing veyepar tool to massage the data.

I recommend someone talking to him about this.

I also recommend that if people would like to see more or different information coming from veyepar that they talk to Carl to see what can happen.

It would be definitely better to use veyepar for getting the data if available since sometimes he'll update the youtube channel if someone submits a request to update information or whatnot.

[codersquid]

This is an example of what he's done to provide pyvideo-shaped data.

http://veyepar3.nextdayvideo.com/main/M/pyvid_json.urls?client=pyohio&show=pyohio_2016

[zerok]

@codersquid The JSON provided there is still quite a bit different from what we have in pyvideo right now :/ Also there are links there that point to pyohio 2015 instead of 2016.

[zerok]

Ah, no, the category field is just wrong in many of these files :(

[codersquid]

yeah, this is why someone should talk to Carl about it. It will be easy for him to fix, or someone to do a pull request.

[codersquid]

For the old pyvideo, whenever Carl or Ryan did a conference, they could push things directly to pyvideo and the api allowed them to push to a category. If someone could work out a similarly automated path (for new content and updates) then for a lot of python conferences you'd have easy access to data.

[xfxf]

@codersquid I'm going to try to tackle this at the sprints as I'm the best positioned person here to fix this. It's V useful for me (I record Pycon Australia + Kiwi Pycon with Veyepar)