polite
polite copied to clipboard
Future ideas
- [ ] Function for creating turn-key scraping bots within docker container: accepts url, how often it should be monitored, where to save results (DB, s3bucket).
- [ ] “polite POST”?
- [ ] can it be extended to enforcing polite behavior on server side, i.e. running alongside, say, plumber or shiny (other R operated web services)?
- [ ] could there be python project based on
requests
,ratelimit
, and memoization withfunctools.lru_cache
- [ ]
use_manners
for API wrappers - [ ] memoize to disk
- [ ] credentials management helpers, if any?