gnomad-browser icon indicating copy to clipboard operation
gnomad-browser copied to clipboard

Certain pages fail when redis is configured but unavailable

Open sjahl opened this issue 1 year ago • 1 comments

When standing up a new demo that lacked redis, but had the redis URL defined in its pod environment variables, some pages seemed fine, but we found that other pages failed to load, and logged timeouts related to the redis cache.

Initial diagnosis here is that we have something (i.e. the GeneCoverage query, when loading the gene page) that is either:

  • not failing gracefully when it has a timeout connecting to redis
  • or, is waiting too long to try and connect to a dead/nonexistent redis and then timing out before it can retrieve the data from elasticsearch.

This would be a good case to try and resolve, so that the app is at least somewhat resilient to failures of the redis cache, and has the ability to run without it. Or, we could fail to start up if a cache is configured but cannot be used.

sjahl avatar Oct 13 '23 19:10 sjahl

From browser meeting:

  • We should fail more gracefully when the cache is unavailable
  • There's a real prod use case of when redis is down, we'd like to be able to return things
  • Any query that uses withCache() will hang for 30 seconds then almost certainly fail if there's no redis

rileyhgrant avatar Nov 16 '23 20:11 rileyhgrant