reddit2telegram
reddit2telegram copied to clipboard
Fillll
Underlying mechanics
Hi, I wanted to ask how do you scrape the posts from Reddit, consider the future price changes for the API? Do you still use the API or do you scrape the HTML with something Selenium or BeautifulSoup? Thanks in advance.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi, I wanted to ask how do you scrape the posts from Reddit, consider the future price changes for the API? Do you still use the API or do you scrape the HTML with something Selenium or BeautifulSoup? Thanks in advance.
I consider continuing to use API. Any concerns?
Isn't the price going to be extremely High?
Il Dom 25 Giu 2023, 16:20 Fillll @.***> ha scritto:
Hi, I wanted to ask how do you scrape the posts from Reddit, consider the future price changes for the API? Do you still use the API or do you scrape the HTML with something Selenium or BeautifulSoup? Thanks in advance.
I consider continuing to use API. Any concerns?
— Reply to this email directly, view it on GitHub https://github.com/Fillll/reddit2telegram/issues/235#issuecomment-1606110595, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSSVFDHQPQULRAGSBS2R73XNBCLZANCNFSM6AAAAAAZTERXEY . You are receiving this because you authored the thread.Message ID: @.***>
I hope we'll fit in 100 requests per minute. Otherwise, yes, extremely high. :(
I understand. I Will probably for my own project develop a scraper library. If you want to collaborate, hit me up.
That will be interesting. Any chance you share your work? Will I be able to use it for this project?
On Jun 25, 2023, at 10:39 AM, djtech-dev @.***> wrote:
I understand. I Will probably for my own project develop a scraper library. If you want to collaborate, hit me up.
— Reply to this email directly, view it on GitHub https://github.com/Fillll/reddit2telegram/issues/235#issuecomment-1606116741, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAE3M3J3BHU7AZ7PKRVOM63XNBESVANCNFSM6AAAAAAZTERXEY. You are receiving this because you commented.
As all my projects, it will be released under the MIT License, so yes.
The only problem is time; i'm already working on a big project in Rust, so I don't have really that much time; i'll probably just develop a basic version (basic support for posts scraping) and try to find a mantainer.
Beacuse for my project only a basic feature (get top posts from a subreddit, media/video can be ignored, but an implementation for media support is a good idea) is needed; so I don't know how "complete" the first version will be, but I think that something like your project can work with no problems with the feature described above.