annotation caching

[vladak]

Observing the latencies for various requests, annotate view often stands out. We should consider caching annotations' results, similarly to xref/historycache, except this would be lazy approach only - once an annotation is displayed it should be cached. This should probably be done only for the latest revision.

[vladak]

There are some technical issues that need to answered like:

• whether to cache the whole page or just the Annotation object
• tunable to enable/disable the caching, perhaps also on per project level (and its default value, should be on by default I think)
• invalidation: possibly during reindex but may be done automatically when stale annotation cache is detected in webapp when performing cache lookup
  • this can use the revision ID stored in the index or hook into the history cache update
• cleanup of empty directory trees will have to be done (like in PendingFileCompleter)

[vladak]

fixed via #4059