minet
minet copied to clipboard
medialab
A webmining CLI tool & library for python.
- filters (name, creator...) - music (name, creator, duration, start_time, info on artist...) - mashup (info on original media...), - media_id, - videos_duration, - video_title, - video_view_count
`Retrieving posts: 1 hashtags [29:45, 1785.08s/ hashtags, posts=71574]Traceback (most recent call last): File "/home/lauramiguel/.pyenv/versions/3.8.13/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/lauramiguel/.pyenv/versions/3.8.13/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals)...
Video links are retrieved when scrapping user profiles on instagram, but not when searching with a hashtag, ony the cover images are retrieved. (Or it seems so actually). Is it...
- [x] request should accept a timeout - [x] Response should accept an already decoded text - [ ] rewire timeout kwarg for playwright crawler - [x] add callback to...
Also refactor the worker to use the Crawler's method and put the related code within the default method?