Stackoverflow-Analysis icon indicating copy to clipboard operation
Stackoverflow-Analysis copied to clipboard

Implemented Caching in loaded datasets for Improved Performance

Open J-B-Mugundh opened this issue 1 year ago • 1 comments
trafficstars

Description:

This PR introduces caching mechanisms for loaded datasets and summary-based functions to enhance application performance and reduce load times. By leveraging Streamlit's built-in st.cache_data function, we aim to optimize repeated access to static CSV data and eliminate unnecessary reloading.

Changes:

  • Implemented a new caching function: cache_data using Streamlit’s st.cache_data feature.
  • Considered implementing a custom caching solution but opted for Streamlit’s in-built function due to its simplicity and efficiency.

Impact:

  • Performance Boost: By caching static datasets (CSV) and summary functions, load times will be significantly reduced, especially in scenarios where real-time data is not required.
  • Resource Efficiency: Reduces redundant processing by storing results from repetitive dataset loading and computation tasks.

References:

https://docs.streamlit.io/develop/api-reference/caching-and-state/st.cache_data

image

Comparison videos of existing load and new load time after cache_data:

Existing: https://drive.google.com/file/d/1DhsmF-K-hCKsOF3mAHOdkFp8q8aB2rqX/view?usp=sharing

New: https://drive.google.com/file/d/1ZF7YFSDCMZ2Y0K9ZTMGbYY2l0nuY6hi8/view?usp=sharing

Related issue:

#372

J-B-Mugundh avatar Oct 05 '24 10:10 J-B-Mugundh