marvin
marvin copied to clipboard
sdss
current top 10 issues on Sentry
In the free mode of Sentry, we have an event history of 7 days after which items get removed. We haven't been resolving issues, so we've lost a lot of information. We were gifted 5000 free events until our cycle resets (the 17th). We can currently see some things again. Here are the top ten issues in Sentry:
Issue Title, Event Count, Estimated Date Range
- 'this is the end of the road. Try using some reasonable inputs.', 79583, year+ - 9 minutes ago
- 'no DB connection found', 8128, year+ - 30 minutes ago
- 'sdss_access is not installed', 2810, 9 months - 1 hour ago
- 'BrainError: Requests Http Status Error: 429 Client Error: TOO MANY REQUESTS for url: https://api.sdss.org/mar...', 2662, 5 months - 17 minutes ago
- 'No local database found. Query cannot be run in local mode', 2440, year+ - 3 days ago
- 'BrainError: Something went wrong on the server side: Failed to retrieve maps 10001-12703: Specified bintype VOR10...', 250, year+ - 2 days ago
- 'NoResultFound: No row was found for one()', 105, 7 months - 3 days ago
- 'get_nsa_data: cannot find NSA row for mangaid=1-628628', 88, 5 months - 4 days ago
- 'OSError: write error', 79, 2 months - 5 hours ago
- 'get_nsa_data: cannot find NSA row for mangaid=1-389720', 52, 6 months - 4 days ago
Some of these are just localhost and/or dev errors before I turned on that filter. Some of these are internal errors captured normally by the code by users. Some may indicate real bugs, e.g. the top two issues. If Marvin is working correctly these are endpoints that should never be triggered. Have a look here for the full list. https://sentry.io/manga/marvin/dashboard/
Having a quick look at this, I'm fairly sure the first two are not issues, but Sentry reporting raised exceptions that are caught in a try-except. Every time you use the decision tree it uses the local mode first. If that fails it raises the "this is the end of the road message", which is caught and is the indication to try remote. Which means that we have had 80k requests for data ... not bad. The second is similar, I think. The third may be similar, if we raise and error while trying to check whether sdss_access is installed and, if it is not, fall over to the extern package. I think the "no local database found. Query cannot ..." may fall in the same category. The others seem legit but mostly related bad input parameters.
The "no DB connection found" comes from mangaid2plateifu, btw, and it is try-except'd.
80k in the last year at least. This makes sense, given how we utilize Sentry. If we don't want to capture these errors, we should reconsider when we send an error to Sentry. If we detach Sentry from explicitly raised MarvinErrors, then I think we'll only get uncaught exceptions, which is maybe what we ultimately care about. Or we sign up and set up an Inbound Filter to remove these particular errors.
If there is a way of filtering these by the string of the message then I think that will work. There are probably only a handful of these that cause most of the problems. The others are probably valid errors. Another thing is to differentiate errors due to invalid inputs and real errors but that may be harder to do.
Also, if we set an Inbound Filter, will the excluded messages count towards the limit of messages in Sentry?
The free Sentry provides the ability to filter out errors
- caused by browser extensions
- coming from localhost servers
- coming from legacy browsers
- coming from web crawlers
- specific IP addresses
On a paid plan we can filter out errors with specific messages and/or specific software releases. I imagine this would open up if we get approved for access as an open source project.
Messages excluded using Inbound Filters do not count towards the limit of Sentry events. So the more filters we apply that can cut out the crap we know we don't want, the more real errors we'll get.