Stackoverflow-Analysis
Stackoverflow-Analysis copied to clipboard
Implemented Caching in loaded datasets for Improved Performance
trafficstars
Description:
This PR introduces caching mechanisms for loaded datasets and summary-based functions to enhance application performance and reduce load times. By leveraging Streamlit's built-in st.cache_data function, we aim to optimize repeated access to static CSV data and eliminate unnecessary reloading.
Changes:
- Implemented a new caching function: cache_data using Streamlit’s st.cache_data feature.
- Considered implementing a custom caching solution but opted for Streamlit’s in-built function due to its simplicity and efficiency.
Impact:
- Performance Boost: By caching static datasets (CSV) and summary functions, load times will be significantly reduced, especially in scenarios where real-time data is not required.
- Resource Efficiency: Reduces redundant processing by storing results from repetitive dataset loading and computation tasks.
References:
https://docs.streamlit.io/develop/api-reference/caching-and-state/st.cache_data
Comparison videos of existing load and new load time after cache_data:
Existing: https://drive.google.com/file/d/1DhsmF-K-hCKsOF3mAHOdkFp8q8aB2rqX/view?usp=sharing
New: https://drive.google.com/file/d/1ZF7YFSDCMZ2Y0K9ZTMGbYY2l0nuY6hi8/view?usp=sharing
Related issue:
#372