4cat icon indicating copy to clipboard operation
4cat copied to clipboard

Bitchute datasource does not currently work

Open dale-wahl opened this issue 1 year ago • 1 comments

Currently BitChute datasets fail to complete. It appears to be some issue with authentication and our current method is no longer working. I have explored the csrftoken token and recreating the cookies but cannot seem to effectively recreate the request to get the proper JSON result.

Failed BitChute request to https://www.bitchute.com/api/search/list/ 3 times, aborting
Stack trace:
threading.py:930 → threading.py:973 → worker.py:117 → processor.py:189 → search.py:84 → search.py:255 → search_bitchute.py:279 → search_bitchute.py:579

Possible methods forward:

  • we rewrite the datasource
    • try to scrape the results from a different request if no one can figure out how to recreate that particular request
    • could use a different scraper (e.g., merge tracker-tracker branch)
  • we deprecate bitchute as a datasource 😢

In the meantime, I have deactivated the datasource from our 4CAT instances.

dale-wahl avatar Jan 17 '24 14:01 dale-wahl

I tried rewriting it to not trigger the token check and failed, maybe it would work when using Selenium, but I'm not sure that's worth it...

stijn-uva avatar Jan 17 '24 15:01 stijn-uva