comses.net icon indicating copy to clipboard operation
comses.net copied to clipboard

Codebase Contributor list caching or querying

Open cpritcha opened this issue 7 years ago • 1 comments

Codebase all_contributors is currently a computed property where a codebase is considered to have a contributor if any of the codebase's release has a contributor. This generates many queries on the codebase list and member profile detail pages.

To reduce the number of queries we could replace all_contributors property with a cache all_contributors related name on the codebase table (a many to many between codebase and contributor).

Alternatively, we could use prefetch_related('releases__codebase_contributors__contributor__user') and prefetch_related('releases__codebase_contributors__contributor__contributor_tags__tag') and then computing all_contributors using the prefetched data but that would be joining quite a few tables together.

I'm leaning towards creating through table between contributors and codebase that can be computed from codebase release and contributor relationships because it makes querying more understandable. Any thoughts @alee?

cpritcha avatar Feb 03 '18 00:02 cpritcha

Caching sounds like a good idea. Instead of caching in the db table, we could also do a lazy cache in redis. Cache invalidation is a problem, but we could have a simple policy of update the cache after a contributor write, or do a very loose eventually consistent policy and set it up on a nightly cron job

alee avatar Feb 03 '18 04:02 alee