Dracovian

Half second timeout not enough to prevent HTTP 429.

3

The script is still prone to being ratelimited. So a workaround to this will be setting the timeout to one second instead of a half second. This should be rectified...

Cache JSON response data for faster updating.

4

Currently the script does the following steps: 1. Read from the config data to gather the guild and channel IDs that are to be scraped. 2. Form a simple search...

* [x] Implement a function to retrieve the latest post in a channel to alleviate lost time when grabbing from inactive channels. * [x] Rename **gatherJSONData** in config file to...

Error messages showing up in multiples when scraping multiple channels per guild.

1

This is likely due to the fact that I haven't forced the script to swallow exceptions in the function named *handleNetworkData* in the main script file. I'll keep debugging the...

Embedded videos that aren't proxied by Discord will point to the source website and thus the methodology of scraping non-proxied embedded content is different than scraping proxied embedded content. An...

I do notice that the popular youtube-dl script integrates with FFMPEG if it can detect it on the host machine (path variables are set so that the FFMPEG binaries can...

To put it simple, those pages were written before the experimental and experimental_requests branches were around. Those pages were written before I started writing my own custom modules for this...

So I'm going to be delving into adding commandline flags to generate a configuration for the script upon the first run. If I ever plan on writing a GUI frontend...

The requests library isn't asynchronous, so that means I'll have to implement aiohttp in future releases. I'll also have to look into the ratelimiting issue but Discord was gracious enough...

This is likely caused by the missing "Accept-Encoding: gzip" header from each request. Cloudflare returns this response header when it errors out: "Vary: Accept-Encoding". The Vary header is used to...