Performance issues on enterprise pages

[filipefurtad0]

Description

The enterprise pages perform poorly and have very high response times (see distribution, pics below). This concerns:

• /admin/enterprises ~~(less critical)~~ -> more impact (10-20 times more traffic)
• /admin/enterprise_relationships

For more context see:

• this thread on Slack
• this discourse post on performance as an acceptance criteria

Expected Behavior

Pages should have a T = 4s.

Actual Behaviour

T(p50) is about ca. 8.5 s for /enterprise_relationships page; if the user is logged in as superadmin response times of over 50s have been observed (seen on p99 percentile).

Steps to Reproduce

1. Best perceived logged in as superadmin on a large instance
2. Open the browser console and visit the relevant pages:
3. /admin/enterprises
4. /admin/enterprise_relationships
5. See also Datadog: i) env:production service:railsoperation_name:rack.requestresource_name:"Admin::EnterprisesController#index" ii) env:production service:railsoperation_name:rack.requestresource_name:"Admin::EnterpriseRelationshipsController#index"

Animated Gif/Screenshot

https://app.datadoghq.com/apm/resource/rails/rack.request/9db53a77925601d7?query=env%3Aproduction%20service%3Arails%20operation_name%3Arack.request%20resource_name%3A%22Admin%3A%3AEnterpriseRelationshipsController%23index%22&env=production&index=apm-search&topGraphs=latency%3Aapdex%2CbreakdownAs%3Apercentage%2Cerrors%3Acount%2Chits%3Acount&start=1625938577938&end=1628530577938&paused=false

https://app.datadoghq.com/apm/resource/rails/rack.request/bca7507e43c9b020?query=env%3Aproduction%20service%3Arails%20operation_name%3Arack.request%20resource_name%3A%22Admin%3A%3AEnterprisesController%23index%22&env=production&index=apm-search&maxPercentile=100&topGraphs=latency%3Alatency%2CbreakdownAs%3Apercentage%2Cerrors%3Acount%2Chits%3Acount&start=1625939118055&end=1628531118055&paused=false

Workaround

Waiting til the content is loadable and the page is usable...

Severity

bug-s3: a feature is broken but there is a workaround

Your Environment

• Version used: not introduced recently; the data above is from the last month so v3.8.5 - v3.8.8

Possible Fix

Possible fixes could be:

• introducing pagination
• reducing data loading with Angular

Issue list

• [ ] #8028

[Matt-Yorkley]

Notes:

• Work on these two pages should be tackled in separate PRs
• I'd probably prioritise /admin/enterprises first, as it gets 10-20 times more traffic and it should be much easier to fix