cugetreg
cugetreg copied to clipboard
[BUG] Investigate random scraper freeze
Problem
Sometimes, the scraper just get randomly stuck (like LITERALLY stuck, no errors, no alert, nothing). This causes the courses to be outdated without any of us knowing.
Task Description
So either we should:
- Investigate the problem and fix it or
- Implement a discord alert (or auto-restart the service) if we detect the scraper is stuck (we should probably do this too anyways).
Additional Context
Right now, we can detect using isScraping property in ScraperService, which if true, would deny scrape action when scrape() is called by scheduler.
Related Teams
- [ ] Frontend
- [X] Backend
- [ ] Data
- [ ] Design
- [ ] Infra
- [ ] QA
Task Advisors
@bombnp
Actually, im not sure if things will change with @OnFireByte 's new scraper. Will investigate later if it still persists.