apexcharts-card icon indicating copy to clipboard operation
apexcharts-card copied to clipboard

Massive resource use

Open tomlut opened this issue 3 years ago • 43 comments

Checklist

  • [x] I updated the card to the latest version available
  • [x] I cleared the cache of my browser
  • [x] I verified that I'm really running the lastest version in my browser console
  • [x] I checked if there is another issue opened with the same problem

Describe the bug I've been quite impressed with the look and function of apexcharts and was going to replace all my mini graph cards but I have found that apexcharts is far too resource intensive to meet my needs.

For example, the bar charts in the screenshot below are daily maximums or last values for the past week. There could be thousands of measurements a day from the inverter that need to be processed.

When loading, the charts grind not only the frontend to a halt but the backend too. I get loads of errors that unrelated integrations are taking too long to respond.

This is with Home Assistant on an i7 mini PC.

I tried setting the graph update to 5 minutes rather than on sensor change. This helped a bit. Except when trying to edit the cards. If the card is still loading data when I try to save it seems to start loading and processing the data again, without cancelling the previous load/process operation. Leading to massive resource use. Closing the page does not help. All the processes continue to run to completion. This pretty much prevents Home Assistant from doing anything else.

I've tried removing the "experimental" colour thresholds. The massive resource use continues to occur.

The mini-graph card (for all it's other failings) does not do this. I have no idea how or why. Caching perhaps?

Version of the card Version: v1.9.0

To Reproduce This is the configuration I used:

type: custom:apexcharts-card
apex_config:
  dataLabels:
    background:
      enabled: false
    style:
      colors:
        - var(--primary-text-color)
graph_span: 7d
header:
  show: true
  title: Daily Energy
series:
  - entity: sensor.energy_inverter_daily
    type: column
    show:
      datalabels: true
    group_by:
      func: max
      duration: 1d

Screenshots If applicable, add screenshots to help explain your problem.

Screenshot 2021-09-15 at 10-35-01 Overview - Home Assistant

Expected behavior A bit of optimisation, particularly when editing the card so that it does not bog down the entire system.

Desktop (please complete the following information):

  • Browser Firefox
  • Version 91.0.2

Smartphone (please complete the following information):

  • Device: [e.g. iPhoneX]
  • OS: 14.8
  • Browser saffari
  • Version ??

Additional context Add any other context about the problem here.

tomlut avatar Sep 15 '21 01:09 tomlut

Caching is disabled while editing the card from the frontend, that's on purpose. Maybe there are some bugs there and I could make it better but I don't use the frontend editor so I never really checked.

With regards to the backend being slow, unfortunately that has nothing to do with the card but to requesting history data (almost exact same code as mini-graph-card). I have an i7 NUC to run home-assistant and I don't experience what you are seeing (and I also load a lot of history data). Once the card is configured, you shouldn't have any issue anymore as caching is enabled again automatically.

With regards to the frontend being slow, all the math is done in frontend but I do not experience that either (neither does a lot of other people using the card). If you hace a slow device for your frontend through, that is quite expected (eg: slow Android tablet, old iPad or equivalent)

RomRider avatar Sep 15 '21 08:09 RomRider

(almost exact same code as mini-graph-card).

That's weird because I don't see the same behaviour with the mini-graph card. I can interrupt it loading to my hearts content when editing via the UI.

Once the card is configured, you shouldn't have any issue anymore as caching is enabled again automatically.

As I said, it's better, but if you interrupt the loading/processing it begins again without (apparently) stopping the previous processes. And that's what gets heavy. It starts trying to load all this data 2 or 3 time in parallel.

If you hace a slow device for your frontend through,

No, it's an older but still quite powerful PC.

tomlut avatar Sep 15 '21 09:09 tomlut

I have found that the graphs were logging lots of errors in the browser console. Seems like this was causing slow loading of the graphs for me. Disabling adblock for the site removed the errors and improved the performance noticeably.

vadipp avatar Oct 04 '21 12:10 vadipp

Sorry, it turns out that the errors in the console are just transient. Disabling adblock didn't affect them. The errors just disappeared, and then magically reappeared again. Seems like I'm actually facing bug #17.

vadipp avatar Oct 05 '21 05:10 vadipp

@vadipp, you can ignore the errors in the console as those are related to some bugs in apexcharts.js but are not affecting functionalities.

That's weird because I don't see the same behaviour with the mini-graph card. I can interrupt it loading to my hearts content when editing via the UI.

I don't understand what you mean? If there is a bug with editing the card in the UI (like having ghost instances of the card still running), it should go away once you close the editor and reload the page.

RomRider avatar Oct 05 '21 07:10 RomRider

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] avatar Nov 05 '21 02:11 github-actions[bot]

Still an issue on 2021.11.1

tomlut avatar Nov 05 '21 10:11 tomlut

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] avatar Jan 02 '22 02:01 github-actions[bot]

Still an issue on 2021.12.7

tomlut avatar Jan 02 '22 02:01 tomlut

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] avatar Feb 03 '22 02:02 github-actions[bot]

Still an issue on 2022.2.0

tomlut avatar Feb 03 '22 02:02 tomlut

I have an interface with two graphs - its super slow. I wish it would generate the graph serverside as an image so it would be super fast for the client!

Hamsterman avatar Feb 23 '22 07:02 Hamsterman

Using last version, I thinks there's some memory leaks with the card, even with only 2 sensors as inputs, massive CPU/RAM usage, as soon as the card is removed memory goes back to normal usage, and it's even worse when editing (most likely due to cache being disable as you mentionned)1

Screenshot_20220302_191609(1)

sicknesz avatar Mar 02 '22 18:03 sicknesz

There might be some issues with edit, yes and I need to have a look at it. however outside of editing the card, the card is resource heavy especially if you have to display A LOT of datapoints (like using a sensor updated every minute or less with raw data over a long period of time).

RomRider avatar Mar 02 '22 18:03 RomRider

There might be some issues with edit, yes and I need to have a look at it. however outside of editing the card, the card is resource heavy especially if you have to display A LOT of datapoints (like using a sensor updated every minute or less with raw data over a long period of time).

I have a sensor with lots of datapoints - what can you recommend me do instead. Also I have the same data in a SQL database.

Hamsterman avatar Mar 03 '22 09:03 Hamsterman

I thought I was losing my mind... I've chased this for almost a month and finally realized that Apex charts IS THE CULPRIT. it started a few core updates ago, basically the charts are so slow as to render them totally unusable. The Ui freezes completely for anywhere from 10-30 seconds, only on Lovelace tabs that have the charts.

Bummer. It was a nice card. Anyone found a substitute card? Fwiw, I'm on core-2022.3.7 now, but it's been going on for weeks. Jeff

jazzmonger avatar Mar 26 '22 02:03 jazzmonger

The mini graph card seems to handle large data sets better. The graphs aren't as customisable or nice though.

tomlut avatar Mar 26 '22 02:03 tomlut

I thought I was losing my mind... I've chased this for almost a month and finally realized that Apex charts IS THE CULPRIT. it started a few core updates ago, basically the charts are so slow as to render them totally unusable. The Ui freezes completely for anywhere from 10-30 seconds, only on Lovelace tabs that have the charts.

Bummer. It was a nice card. Anyone found a substitute card? Fwiw, I'm on core-2022.3.7 now, but it's been going on for weeks. Jeff

@jazzmonger, I'm experiencing the same (UI freezes for many seconds and I can't even switch tab in the dashboard until the graph is loaded) and I'm moving toward Plotly. It seems a younger project but it's very powerful, lighting fast, and you can also interact with the graph. It is not a direct drop-in replacement, but I'm moving to it. The worst issue for me is dbuezas/lovelace-plotly-graph-card#34. But considering the drawbacks of apex, it's nothing.

zanna-37 avatar Mar 26 '22 07:03 zanna-37

Apex charts IS THE CULPRIT

I have a dashboard with 100 charts displayed at once and it's working fine. Just saying there's an issue without providing any data/performance graph isn't going to help with fixing a potential issue.

RomRider avatar Mar 26 '22 11:03 RomRider

Yeah but how many data points do each of your graphs use and what features of Apexcharts are you using?

There is definitely an issue using multiple graphs when using sensors that can update every second or so (like from my solar inverter).

tomlut avatar Mar 27 '22 01:03 tomlut

Yeah but how many data points do each of your graphs use and what features of Apexcharts are you using?

There is definitely an issue using multiple graphs when using sensors that can update every second or so (like from my solar inverter).

Yeah, me too. I'm visualizing 10h of data with 8/9 traces that update every minute.

zanna-37 avatar Mar 27 '22 09:03 zanna-37

Use update_interval and maybe an aggregate (eg: avg) function over that then to reduce the sample rate to 1min, updating every second is way too high. Every seconds over 10h is 36000 data points, not surprising it is quite slow...

RomRider avatar Mar 27 '22 10:03 RomRider

I wasn't aware of the update interval. I'll give it a go. Yeah I didn't write the SMA Integration that does that.

tomlut avatar Mar 27 '22 14:03 tomlut

Use update_interval and maybe an aggregate (eg: avg) function over that then to reduce the sample rate to 1min, updating every second is way too high. Every seconds over 10h is 36000 data points, not surprising it is quite slow...

update_interval does not work for me since the unresponsive graph happens on page refresh. For the average, I have a 4-minutes window and it is still slow. By the way, I moved to Plotly and it works nicely.

zanna-37 avatar Mar 31 '22 20:03 zanna-37

With mini-graph you can select points_per_hour, this helps alot with the speed of the graph.

I have several sensors updating every second, this really slows everything down when trying to use apex. Every page load takes a lot of resources from both backend and browser, even with group_by and func: avg.

I think a solution would be if you specify group_by and func: first, that should only fetch the first value from the database, not all of them and then filter out what to show in the browser. Another solution would be to be able to specify a value on how many data points you want to fetch, similar to points_per_hour.

Also: update_interval isn't working. https://github.com/RomRider/apexcharts-card/issues/293

tonyaik avatar Apr 07 '22 14:04 tonyaik

For anyone having this issue, I just updated to v1.11.0-dev.1, my resource usage is way down at the moment. I will keep monitoring. But try out v1.11.0-dev.1 if you're having these issues.

tonyaik avatar Apr 12 '22 10:04 tonyaik

For those who don't have update_interval working, could you please share your config in https://github.com/RomRider/apexcharts-card/issues/293 as I can't reproduce the issue.

RomRider avatar Apr 19 '22 15:04 RomRider

update_interval fixed in 2.0.0

RomRider avatar Apr 25 '22 02:04 RomRider

The issue go much worse for me in 2.0.0. It was very slow, taking up to 30 seconds to load on previous versions, but now it's unusable.

My update_interval is 2 hours and I can't even get the first card to load before it crashes. I managed to get it to loaded after 10 attempts and clicking Wait several times. This is how the graphs should look like image Is there anything I can optimize in my config to improve the situation: image image

convicte avatar Apr 27 '22 09:04 convicte

The issue was very bad for me with 2.0.1 as well. But it magically became better. Try clearing chrome cache etc.

tonyaik avatar Apr 27 '22 09:04 tonyaik