openems icon indicating copy to clipboard operation
openems copied to clipboard

Improve Edge Caching Mechanism in InitializeEdgesWorker

Open Sn0w3y opened this issue 1 year ago • 8 comments

This pull request enhances the InitializeEdgesWorker class by implementing a regular caching mechanism for edges from PostgreSQL. The main improvements are:

Regular edge caching: Edges are now cached immediately upon startup and then every 20 minutes. Simplified scheduling: Uses a ScheduledExecutorService to manage both immediate and periodic tasks. Executor management: Ensures that executors are properly shut down when stopping the worker. Improved error handling and logging: Enhanced logging for better error tracing and debugging.

Benefits

  • Regular Edge Caching: Keeps the edge cache updated by regularly fetching data from PostgreSQL.
  • Simplified Scheduling: Reduces complexity by using a single scheduler for task execution.
  • Resource Management: Ensures executors are properly shut down to prevent resource leaks.
  • Improved Logging: Enhanced error messages facilitate easier debugging.

Sn0w3y avatar Oct 01 '24 11:10 Sn0w3y

Codecov Report

:x: Patch coverage is 0% with 27 lines in your changes missing coverage. Please review.

:x: Your patch status has failed because the patch coverage (0.00%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #2827      +/-   ##
=============================================
- Coverage      57.01%   56.98%   -0.03%     
+ Complexity      9659     9649      -10     
=============================================
  Files           2262     2262              
  Lines          96538    96556      +18     
  Branches        7126     7128       +2     
=============================================
- Hits           55031    55011      -20     
- Misses         39460    39498      +38     
  Partials        2047     2047              
:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Oct 01 '24 11:10 codecov[bot]

@michaelgrill: Can you please take over?

~~Should we switch this caching to a library that handles updating internally? Guava provides a nice Cache feature: https://github.com/google/guava/wiki/cachesexplained~~ Reconsiderung: no, this does not really make sense here.

@Sn0w3y: I suggest we make the refresh time configurable (time in minutes). What do you think?

sfeilmeier avatar Oct 13 '24 13:10 sfeilmeier

@michaelgrill: Can you please take over?

~Should we switch this caching to a library that handles updating internally? Guava provides a nice Cache feature: https://github.com/google/guava/wiki/cachesexplained~ Reconsiderung: no, this does not really make sense here.

@Sn0w3y: I suggest we make the refresh time configurable (time in minutes). What do you think?

I will add a Configurable Time to the Worker aswell as "Enable and Disable" Option :)

Sn0w3y avatar Oct 13 '24 14:10 Sn0w3y

I will add a Configurable Time to the Worker aswell as "Enable and Disable" Option :)

One value is sufficient. Negative or zero means "disable".

sfeilmeier avatar Oct 13 '24 14:10 sfeilmeier

I will add a Configurable Time to the Worker aswell as "Enable and Disable" Option :)

One value is sufficient. Negative or zero means "disable".

Done :)

Sn0w3y avatar Oct 13 '24 14:10 Sn0w3y

I had a quick look and renamed the config property to cacheRefreshInterval. But InitializeEdgesWorker also has to be renamed, because the name does not make any sense anymore. Also Checkstyle currently fails...

sfeilmeier avatar Oct 13 '24 16:10 sfeilmeier

@Sn0w3y Thank you so much! We’ve started testing it internally!

katsuya avatar Dec 02 '24 22:12 katsuya

@Sn0w3y Thank you so much! We’ve started testing it internally!

Does it work? Could you please tell us if the PR is successful ?

Sn0w3y avatar Jan 05 '25 04:01 Sn0w3y

This PR has been automatically marked as stale due to inactivity. It will be closed in 7 days if no further activity occurs.

github-actions[bot] avatar Sep 18 '25 07:09 github-actions[bot]

This PR has been closed due to inactivity

It was automatically closed because there has been no recent activity. If the PR is still relevant, please feel free to reopen and update it and add any new information.

github-actions[bot] avatar Oct 09 '25 02:10 github-actions[bot]