Medusa icon indicating copy to clipboard operation
Medusa copied to clipboard

Published 20 hours ago • verified publisher icon pymedusa

Poor performance / page load times of /home

Open sammcj opened this issue 3 years ago • 50 comments

Describe the bug

Loading /home (with the list of shows) takes between 5.2 and 8.5s.

I have been discussing and sharing database and log dumps with @p0psicles on this issue.

To Reproduce Steps to reproduce the behaviour:

  1. Install medusa from source / git on a CentOS 7 host OS and Python 3.6.8 (latest stable python3 for CentOS 7)
  2. Add between 250-350 shows to medusa
  3. Load the home page
  4. Notice that the page takes a long time to finish loading

It does not make any difference which browser you use on the client side but I've tried: Firefox 81 on a new i9 MBP, Chrome stable on the same machine, Safari on iOS 13 and 14 on an iPhone 11 Pro.

Note: I have also had a friend who is able to reproduce the poor performance - although not quite as bad, at around 2.5-3s - running the latest Medusa Docker image and on a wired network with both Firefox and Chrome.

Expected behaviour

The /home page should not take longer than 1-2 seconds at most to load.

Screenshots

  • Shared with @p0psicles

Medusa (please complete the following information):

  • OS: CentOS 7, Kernel-ML 5.8.7-1
  • Branch: Master
  • Commit: b352bb6924afcdfafce176a540d53ce405ca1312
  • Python version: 3.6.8
  • Database version: 10

Debug logs (at least 50 lines): General > Advanced Settings > Enable debug

  • Provided to @p0psicles

Additional context

The more columns you unselect (as pictured below) the faster the page loads, for example with no columns ticked the page loads in just over 1s, but with all ticked it can take as long as 8.5s.

columns

I've tried:

  • Starting medusa with an empty / no main.db: It's faster but still around 2.5s with no shows in the database at all.
  • Starting medusa with an empty config.ini, main.db, cache.db and failed.db: again - faster, but still around 2.5s.
  • I've ensured that the server is not under any load and the I/O is near idle.
  • The server is an i7-4770K, 32GB RAM, Intel NVMe PCIe 750 SSD for OS+apps.
  • The network is wireless, but with excellent signal strength (near the router), high end unifi networking hardware and has a throughput of around 600-850MB/s.
  • All other web apps on the server load much faster with no performance problems (sab, couch, nzbhydra, plex) although none of those are Python3.

firefox_perf_graph_medusa

sammcj avatar Sep 28 '20 06:09 sammcj

Python3 isn't the issue it's a Vue problem. It's being worked on.

OmgImAlexis avatar Sep 28 '20 06:09 OmgImAlexis

@OmgImAlexis we've already have done a lot of tests. And we've eliminated already a lot also. One of the most alarming issues, is that loading his server:8081/home request took 5s. So just loading the network resource. So no js is running at that time.

On my machine (a 10y old 1.6 ghz atom, running at 100% cpu) it takes 200ms. I've looked at his chrome perf report. And allot of the static asset loading takes almost 10x as long compared to mine also.

Event with an empty main.db, it's still taking 2.5s for the /home call.

p0psicles avatar Sep 28 '20 07:09 p0psicles

@sammcj could you do the following. In config -> general -> interface -> Web interface, enable HTTP logs. and restart. Now reload home and check in chrome dev tools, the network tab. Click the /home call. And create a screenshot of the timing tab.

Now in your application.log, find the TORNADO logs for that request. Mine look like:

2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 302 GET /medusa/ (127.0.0.1) 6.39ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/home/ (127.0.0.1) 7.80ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/vendors.css?31419 (127.0.0.1) 5.90ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/vender.min.css?31419 (127.0.0.1) 3.41ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/themed.css?31419 (127.0.0.1) 3.12ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/bootstrap-formhelpers.min.css?31419 (127.0.0.1) 3.38ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/browser.css?31419 (127.0.0.1) 3.11ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/lib/jquery-ui-1.10.4.custom.min.css?31419 (127.0.0.1) 3.15ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/lib/jquery.qtip-2.2.1.min.css?31419 (127.0.0.1) 2.99ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/style.css?31419 (127.0.0.1) 2.98ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/print.css?31419 (127.0.0.1) 2.77ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/css/country-flags.css?31419 (127.0.0.1) 3.04ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/js/vendors.js?31419 (127.0.0.1) 8.14ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/js/vendors~date-fns.js?31419 (127.0.0.1) 2.94ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/js/medusa-runtime.js?31419 (127.0.0.1) 5.38ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/js/index.js?31419 (127.0.0.1) 3.17ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 304 GET /medusa/js/app.js?31419 (127.0.0.1) 2.68ms
2020-09-28 08:59:04 INFO     TORNADO :: [b352bb6] 101 GET /medusa/ws/ui (127.0.0.1) 2.26ms

p0psicles avatar Sep 28 '20 07:09 p0psicles

We're running multiple threads right...?

This kinda points towards a single thread holding all other requests. Otherwise if it's not that then something on the page is preventing anything else from loading until the series endpoint is finished. Are we maybe using await somewhere when .then should be used instead?

Screen Shot 2020-09-28 at 4 38 11 pm

OmgImAlexis avatar Sep 28 '20 07:09 OmgImAlexis

@OmgImAlexis good point. I think we are. But that's kinda a different issue. I'll need to make sure there's not an issue with tornado serving out requests before I hop onto the next. But I appreciate your help!

p0psicles avatar Sep 28 '20 07:09 p0psicles

@OmgImAlexis I know you have a high spec system. How is your performance? Like until the first draw?

p0psicles avatar Sep 28 '20 07:09 p0psicles

Interestingly while this is happening I checked my unraid overview and CPU only goes up 10% and RAM barely moves.

Running a Xeon L5640 @ 2.27GHz so 12 threads.

OmgImAlexis avatar Sep 28 '20 07:09 OmgImAlexis

The whole page is locked until the first draw which happens around 12s.

OmgImAlexis avatar Sep 28 '20 07:09 OmgImAlexis

@p0psicles I loaded the page a few times, it randomly seems slightly faster this morning loading in around 4.5-5.5s, but not doubt it'll get slower randomly again.

TORNADO logs 
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /home/ (192.168.0.1) 1.54ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/vendors.css?24049 (192.168.0.1) 3.57ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/vender.min.css?24049 (192.168.0.1) 0.82ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/browser.css?24049 (192.168.0.1) 0.45ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/bootstrap-formhelpers.min.css?24049 (192.168.0.1) 0.74ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/lib/jquery-ui-1.10.4.custom.min.css?24049 (192.168.0.1) 0.87ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/lib/jquery.qtip-2.2.1.min.css?24049 (192.168.0.1) 0.57ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/themed.css?24049 (192.168.0.1) 0.70ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/index.js?24049 (192.168.0.1) 0.46ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/style.css?24049 (192.168.0.1) 1.65ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/parsers.js?24049 (192.168.0.1) 0.56ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/add-shows/init.js?24049 (192.168.0.1) 0.68ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/add-shows/trending-shows.js?24049 (192.168.0.1) 0.48ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/add-shows/recommended-shows.js?24049 (192.168.0.1) 0.46ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/common/init.js?24049 (192.168.0.1) 0.57ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/notifications.js?24049 (192.168.0.1) 0.45ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/app.js?24049 (192.168.0.1) 0.45ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/browser.js?24049 (192.168.0.1) 0.58ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/country-flags.css?24049 (192.168.0.1) 0.56ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /css/print.css?24049 (192.168.0.1) 0.43ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/vendors~date-fns.js?24049 (192.168.0.1) 0.85ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/lib/formwizard.js?24049 (192.168.0.1) 0.71ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/lib/lazyload.js?24049 (192.168.0.1) 0.62ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/config/init.js?24049 (192.168.0.1) 0.72ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/add-shows/popular-shows.js?24049 (192.168.0.1) 0.43ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /fonts/open-sans-v15-latin-regular.ttf (192.168.0.1) 0.42ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/lib/bootstrap-formhelpers.min.js?24049 (192.168.0.1) 55.82ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /fonts/open-sans-v15-latin-italic.ttf (192.168.0.1) 0.40ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/vender.min.js?24049 (192.168.0.1) 43.29ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/medusa-runtime.js?24049 (192.168.0.1) 60.80ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /js/vendors.js?24049 (192.168.0.1) 70.72ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /images/medusa.png (192.168.0.1) 0.61ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /images/menu/system18.png (192.168.0.1) 0.57ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /images/menu/system18-2.png (192.168.0.1) 0.50ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/config/ (192.168.0.1) 47.85ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/stats/ (192.168.0.1) 46.83ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /fonts/glyphicons-halflings-regular.woff2 (192.168.0.1) 0.46ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/stats/show (192.168.0.1) 78.23ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /images/loading16-dark.gif (192.168.0.1) 0.93ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/series?limit=1000&page=1 (192.168.0.1) 226.63ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /images/ico/favicon-196.png (192.168.0.1) 0.81ms
2020-09-29 09:50:55 INFO     TORNADO :: [b352bb6] 200 GET /images/ico/favicon-32.png (192.168.0.1) 0.60ms
2020-09-29 09:50:57 INFO     TORNADO :: [b352bb6] 200 GET /images/imdb.png (192.168.0.1) 1.54ms
2020-09-29 09:50:57 INFO     TORNADO :: [b352bb6] 200 GET /images/thetvdb16.png (192.168.0.1) 1.28ms
2020-09-29 09:50:57 INFO     TORNADO :: [b352bb6] 200 GET /images/no16.png (192.168.0.1) 0.35ms
2020-09-29 09:50:57 INFO     TORNADO :: [b352bb6] 200 GET /images/yes16.png (192.168.0.1) 0.35ms
2020-09-29 09:50:57 INFO     TORNADO :: [b352bb6] 200 GET /images/tmdb16.png (192.168.0.1) 1.54ms
2020-09-29 09:50:57 INFO     TORNADO :: [b352bb6] 200 GET /images/tvmaze16.png (192.168.0.1) 1.66ms
2020-09-29 09:51:14 INFO     TORNADO :: [b352bb6] 304 GET /home/ (192.168.0.164) 2.45ms
2020-09-29 09:51:14 INFO     TORNADO :: [b352bb6] 101 GET /ws/ui (192.168.0.164) 0.96ms
2020-09-29 09:51:14 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/config/ (192.168.0.164) 48.78ms
2020-09-29 09:51:14 INFO     TORNADO :: [b352bb6] 304 GET /api/v2/stats/ (192.168.0.164) 46.74ms
2020-09-29 09:51:15 INFO     TORNADO :: [b352bb6] 304 GET /api/v2/stats/show (192.168.0.164) 88.63ms
2020-09-29 09:51:15 INFO     TORNADO :: [b352bb6] 304 GET /api/v2/series?limit=1000&page=1 (192.168.0.164) 219.68ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /home/ (192.168.0.164) 3.62ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/vender.min.css?24049 (192.168.0.164) 0.61ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/vendors.css?24049 (192.168.0.164) 7.46ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/bootstrap-formhelpers.min.css?24049 (192.168.0.164) 0.75ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/browser.css?24049 (192.168.0.164) 1.42ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/lib/jquery-ui-1.10.4.custom.min.css?24049 (192.168.0.164) 1.36ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/lib/jquery.qtip-2.2.1.min.css?24049 (192.168.0.164) 0.71ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/style.css?24049 (192.168.0.164) 1.76ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/themed.css?24049 (192.168.0.164) 0.70ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/print.css?24049 (192.168.0.164) 0.44ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /css/country-flags.css?24049 (192.168.0.164) 0.55ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/vendors~date-fns.js?24049 (192.168.0.164) 0.87ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/index.js?24049 (192.168.0.164) 0.49ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/lib/lazyload.js?24049 (192.168.0.164) 0.66ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/lib/formwizard.js?24049 (192.168.0.164) 0.71ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /fonts/open-sans-v15-latin-regular.ttf (192.168.0.164) 0.42ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /fonts/open-sans-v15-latin-italic.ttf (192.168.0.164) 0.41ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/lib/bootstrap-formhelpers.min.js?24049 (192.168.0.164) 33.72ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/parsers.js?24049 (192.168.0.164) 0.50ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/vender.min.js?24049 (192.168.0.164) 51.57ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/config/init.js?24049 (192.168.0.164) 0.49ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/add-shows/init.js?24049 (192.168.0.164) 0.57ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/medusa-runtime.js?24049 (192.168.0.164) 62.35ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/add-shows/popular-shows.js?24049 (192.168.0.164) 0.45ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/add-shows/recommended-shows.js?24049 (192.168.0.164) 0.45ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/add-shows/trending-shows.js?24049 (192.168.0.164) 0.49ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/common/init.js?24049 (192.168.0.164) 0.54ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/browser.js?24049 (192.168.0.164) 0.92ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/notifications.js?24049 (192.168.0.164) 0.56ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/app.js?24049 (192.168.0.164) 0.48ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /js/vendors.js?24049 (192.168.0.164) 90.91ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 101 GET /ws/ui (192.168.0.164) 1.33ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /images/medusa.png (192.168.0.164) 0.60ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /images/menu/system18.png (192.168.0.164) 0.58ms
2020-09-29 09:51:33 INFO     TORNADO :: [b352bb6] 200 GET /images/menu/system18-2.png (192.168.0.164) 0.48ms
2020-09-29 09:51:34 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/config/ (192.168.0.164) 52.05ms
2020-09-29 09:51:34 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/stats/ (192.168.0.164) 50.83ms
2020-09-29 09:51:34 INFO     TORNADO :: [b352bb6] 200 GET /fonts/glyphicons-halflings-regular.woff2 (192.168.0.164) 0.59ms
2020-09-29 09:51:34 INFO     TORNADO :: [b352bb6] 200 GET /images/loading16-dark.gif (192.168.0.164) 0.60ms
2020-09-29 09:51:34 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/stats/show (192.168.0.164) 95.38ms
2020-09-29 09:51:34 INFO     TORNADO :: [b352bb6] 200 GET /images/ico/favicon.ico (192.168.0.164) 0.98ms
2020-09-29 09:51:34 INFO     TORNADO :: [b352bb6] 200 GET /api/v2/series?limit=1000&page=1 (192.168.0.164) 280.79ms

I've sent the profile to you via discord.

sammcj avatar Sep 28 '20 23:09 sammcj

@OmgImAlexis, like you I don't notice any excess CPU utilisation server side either, to me it feels like inefficient javascript around the payload containing the shows.

sammcj avatar Sep 28 '20 23:09 sammcj

@OmgImAlexis regarding threads - if Medusa is using threads - it doesn't seem to do a good job of it as if I do perform a CPU heavy task such as bulk updating, scanning media etc... Python smashes 1 CPU core/thread and does nothing with the other 7.

sammcj avatar Sep 28 '20 23:09 sammcj

Chrome network request timings as requested by @p0psicles, note the page load times are up to around 30s randomly if I use Chrome rather than Firefox (which is still around 6s) right now:

screenshot 2020-09-29 at 3 18 40 pm

sammcj avatar Sep 29 '20 05:09 sammcj

Switched to develop branch.

Here's my unit script:

[Unit]
Description=Medusa Daemon
After=network.target

[Service]
User=apps
Group=apps

Type=simple
ExecStart=/usr/bin/python3.6 /opt/medusa/start.py -q --nolaunch --datadir=/opt/medusa
TimeoutStopSec=25
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target

ps:

~ ps -ef | egrep -i medusa
apps     24049     1  0 09:46 ?        00:02:42 /usr/bin/python3.6 /opt/medusa/start.py -q --nolaunch --datadir=/opt/medusa

htop:

screenshot 2020-09-29 at 3 23 13 pm

sammcj avatar Sep 29 '20 05:09 sammcj

You’re not taking into account that chrome tends to skip caches when the network tab is open. That’s likely why you’re seeing a higher time.

Have you tried the latest develop commit?

OmgImAlexis avatar Sep 29 '20 09:09 OmgImAlexis

True, yeah I don't usually use chrome, I'm always in Firefox or Safari (mobile).

I switched to develop yesterday afternoon, it does actually seem a little quicker at loading pages in general, I'll need to do some more timing tests on /home in FF and chrome today.

sammcj avatar Sep 29 '20 23:09 sammcj

Develop on FF 81:

screenshot 2020-09-30 at 9 52 03 am

I managed to get it to load in 1.5s on Safari (iOS, iPhone 11 Pro) but that was by unticking half the columns.

sammcj avatar Sep 29 '20 23:09 sammcj

I'm also now noticing more 403s on the websockets connection in FF81:

screenshot 2020-09-30 at 9 53 42 am

sammcj avatar Sep 29 '20 23:09 sammcj

What is interesting is that the develop branch seems a lot faster on Chrome now (3.27s):

screenshot 2020-09-30 at 9 56 31 am

sammcj avatar Sep 29 '20 23:09 sammcj

I've done some screen recordings of Medusa reloading the /home page in both Firefox and Chrome, this is running off develop and with cache disabled. I've recorded both the network tab during a load and done a performance profile as well.

These are 4k so the text and graphs should be readable (if youtube doesn't over-compress them).

Firefox:

https://youtu.be/elrOGMTI1ao

Chrome:

https://youtu.be/UzUe8bmd8G4

sammcj avatar Sep 30 '20 00:09 sammcj

Did you ever manage to find a solution for this?

When opening my home page it takes around 23 seconds and this is internal over 1 Gbit network. image

One of the biggest drains seems to be loading the poster images for all shows as they are retrieved in full size for each show. I'm using the Small Poster layout

Rouzax avatar Nov 02 '21 07:11 Rouzax

No, several friends had the same issue and in the end we all moved to sonarr.

On 2 Nov 2021, at 18:57, Rouzax @.***> wrote:

 Did you ever manage to find a solution for this?

When opening my home page it takes around 23 seconds and this is internal over 1 Gbit network.

One of the biggest drains seems to be loading the poster images for all shows as they are retrieved in full size for each show. I'm using the Small Poster layout

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

sammcj avatar Nov 02 '21 08:11 sammcj

@Rouzax that should only be an issue the first time. Also it's lazy loading. So it should only load the posters that are visible on screen. Although i'm not 100% sure about that.

But as said, it should get the poster images from browser cache the next time you open the page. Btw numberous of improvements have been made to try to improve the load speed of the home layout since this issue was openend.

Like we added localStorage caching. Transitioned everything to vue, and a few others. But the issue remains the large amount of vue components it has to create when showing a lot of shows. I can't do much about that.

Maybe @OmgImAlexis can take a look and consult on what can be improved.

p0psicles avatar Nov 02 '21 10:11 p0psicles

@p0psicles it does not appear to use the cached images. Maybe it is an idea to generate and store a thumbnail of the posters in Medusa, this could speed it up.

It does do lazy loading, but still seems to grab the images each time. Medusa is running on hardware that is powerful enough and have the CPU throttling disabled.

CPU

	Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz

	Base speed:	2,00 GHz
	Sockets:	1
	Virtual processors:	12
	Virtual machine:	Yes
	L1 cache:	N/A

	Utilization	7%
	Speed	2,00 GHz
	Up time	0:01:06:20
	Processes	132
	Threads	1632
	Handles	44850

Memory

	16,0 GB

	Slots used:	N/A
	Hardware reserved:	8,0 GB
	Maximum memory:	16,0 GB

	Available	4,6 GB
	Cached	630 MB
	Committed	3,4/10,4 GB
	Paged pool	124 MB
	Non-paged pool	119 MB
	In use (Compressed)	3,3 GB (0 MB)

There is a reverse NGINX proxy in between, but that is also running on the same virtualized hardware.

Here is a screen recording from Firefox from my home page. Did a refresh twice. https://user-images.githubusercontent.com/4103090/139833836-38e4c268-1d4d-4cd5-86f7-8befc247bc69.mp4

EDIT: Second screen recording directly to Medusa without Reverse Proxy https://user-images.githubusercontent.com/4103090/139835052-9698d3d9-d5dc-43ff-bd45-ddc80ff91c2b.mp4

Rouzax avatar Nov 02 '21 10:11 Rouzax

@Rouzax I don't know if it is possible for you, but could you try running Medusa without the reverse proxy? I have the feeling that the going through the proxy is what is causing this.

medariox avatar Nov 02 '21 10:11 medariox

@medariox Was already in the process of capturing that 😄 and it is just as fast/slow Have updated the original post

Rouzax avatar Nov 02 '21 11:11 Rouzax

When looking at the image cache in Medusa, There is a thumbnail folder but most times it just holds the same image size image

EDIT: As a test took all 234 poster images in the thumbnails folder and resized them to max 1024 pixels on the longest side and saved them as jpg. Went from 135MB to 28MB in that folder It does speed up the loading of the posters themselves see the result screen record

https://user-images.githubusercontent.com/4103090/139838730-284dc3a3-6e38-45cb-8db6-a7e7844a2f53.mp4 After wards resized them again and now to 640px and optimized jpg reduced the total size to 12.5MB and that sped up the loading of posters again

EDIT2: Also getting a lot of 404s on images that are not existing but each one will add additonal time image

Rouzax avatar Nov 02 '21 11:11 Rouzax

Last I looked at this it was the JS creating tables inside tables that really slowed things now. 🤷

sammcj avatar Nov 02 '21 11:11 sammcj

@Rouzax but can you check the browser devtools network tab to see if it's getting them from cache?

p0psicles avatar Nov 02 '21 11:11 p0psicles

It's downloading them again. ~~Downloading them again because I have CleanURLs running which blocks etags.~~ ~~https://gitlab.com/KevinRoebert/ClearUrls~~ ~~Have disabled that now, but does not really speed it up~~

image

Rouzax avatar Nov 02 '21 11:11 Rouzax

Screenshot from 2021-11-02 12-39-39 See for me it's not. It's getting them from cache

p0psicles avatar Nov 02 '21 11:11 p0psicles