add some checks to remove assignedData and redis items that shouldn't be there
This is a patch to avoid with issues stemming from a bug which causes the redis queue to desync and still contain items which have been labeled.
NOTE: Any idea for how to test this?
We've tracked down part of the issue to a conflict between gunicorn workers and the init_redis function that happens on application start up. Since there were multiple workers starting up application processes, the init_redis function was being evoked by each of the processes. This lead to duplicate data in the queue.
Since most of the application is built on the assumption that the redis queue holds the most up-to-date ordered, unique, assignable data, this led to a multitude of other errors.
I plan on patching the init_redis duplication by adding a --preload to the production backend run command. Additionally I'll look into moving this patch into a function and adding a simple test for it. That way if "bad" data ever does get stuck in assigned data again, it will not cause the annotation flow to become roadblocked.