Permissions checking is inefficient in dashboards

[macobo]

In what situation are you experiencing subpar performance?

Dashboard APIs do a lot of unneccessary subquerying, leading to poor performance.

Part of this is the permissioning part of these APIs.

How to reproduce

1. Add logging for queries made on a dashboard
2. Load a large dashboard

Details

Methods like get_effective_privilege_level on insights and dashboards are called on every load.

They are passed user-ids and as a result need to frequently query multiple models (team, organization, organization membership, explicit_team_memberships, dashboard counts) to calculate results.

Given this is done many times each time a dashboard is loaded, this leads to performance issues.

Solving this is tricky due to the amount of relations involved. Some options:

1. Explicitly pass (and store) team/org objects in serializer contexts, preload relevant relations and refactor code to use those relations over explicit querying.
2. Load less data about insights on dashboards, refactor frontend navigation

Environment

• [x] PostHog Cloud
• [ ] self-hosted PostHog, version/commit: please provide

Additional context

Stumbled upon this when debugging performance of dashboard APIs.

cc @Twixes as feature author

Thank you for your performance issue report – we want PostHog to go supersonic!

[Twixes]

Would be awesome to know how impactful these inefficencies are by measuring those requests with Sentry tracing and tracking get_effective_privilege_level calls as spans. Can we do that easily?