hoad icon indicating copy to clipboard operation
hoad copied to clipboard

reduce RAM utilisation in shiny app

Open maxheld83 opened this issue 5 years ago • 1 comments
trafficstars

now that we're on GCP (instead of shinyapps.io) as per #25, we're getting a clearer view into the resource utilisation of our shiny app.

It ain't pretty.

It (apparently, pending further testing) won't be stable below 4GiB of RAM.

I am guessing this is mostly the pretty huge data object being loaded into RAM, which is just R design.

Still, I should profile this to be sure that's what's going on.

Long-term, to make the app easier on the RAM, it would be good to offload most of this work to Google BigQuery or a separate R (plumber) API.

(We're not using much CPU, as expected).

maxheld83 avatar Jun 10 '20 15:06 maxheld83

nope it's not the data object, or at least not just them.

pryr::object_size(hoad::hybrid_publications(), hoad::unpaywall_df())

yields 56.9 MB which is not nothing, but doesn't eat up 4GiB of RAM.

Will need to investigate further, later.

maxheld83 avatar Jun 10 '20 15:06 maxheld83