osrd icon indicating copy to clipboard operation
osrd copied to clipboard

WIP: OSRD goes async

Open ElysaSrc opened this issue 1 year ago • 2 comments
trafficstars

⚠️ This pull request will become huge. I'm still rewriting history quite often now. Most of the work in this PR isn't tested yet.

This PR holds the whole modifications for making OSRD async using a RabbitMQ. This is a work in progress, here is an overview:

  • [X] Modify local environement to provide rabbitmq
  • [X] Implement Core Controller
    • [X] Implement Kubernetes driver
    • [X] Implement Docker driver
    • [X] Implement RabbitMQ driver
    • [X] Implement logic
  • [ ] Test the whole stack
    • [X] Against Docker
    • [ ] Against Kubernetes (from within the cluster)
  • [ ] Modify Core to use queues
    • [ ] Create a client library for enqueing/dequeing messages
    • [ ] Use the said library
  • [ ] Modify Editoast to use queues
    • [X] Create a client library for enqueing/dequeing messages
    • [ ] Use the said library

Closes #6679, #6680, #6681

ElysaSrc avatar Apr 04 '24 15:04 ElysaSrc

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 27.97%. Comparing base (fc0cd2e) to head (585fb3a). Report is 41 commits behind head on dev.

:exclamation: Current head 585fb3a differs from pull request most recent head 406f69e. Consider uploading reports for the commit 406f69e to get more accurate results

Additional details and impacted files
@@             Coverage Diff              @@
##                dev    #7103      +/-   ##
============================================
- Coverage     29.00%   27.97%   -1.03%     
  Complexity     2250     2250              
============================================
  Files          1070     1069       -1     
  Lines        132704   135480    +2776     
  Branches       2728     2746      +18     
============================================
- Hits          38487    37901     -586     
- Misses        92637    95981    +3344     
- Partials       1580     1598      +18     
Flag Coverage Δ
core 80.07% <ø> (-0.03%) :arrow_down:
editoast 74.80% <ø> (+0.13%) :arrow_up:
front 8.95% <ø> (-0.35%) :arrow_down:
gateway 2.65% <ø> (+0.19%) :arrow_up:
railjson_generator 87.15% <ø> (ø)
tests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Apr 04 '24 15:04 codecov[bot]

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 10.90%. Comparing base (814e018) to head (70c0eeb). Report is 2 commits behind head on dev.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

:exclamation: There is a different number of reports uploaded between BASE (814e018) and HEAD (70c0eeb). Click for more details.

HEAD has 9 uploads less than BASE
Flag BASE (814e018) HEAD (70c0eeb)
railjson_generator 2 1
gateway 2 1
tests 2 0
front 2 1
core 2 0
editoast 2 0
Additional details and impacted files
@@              Coverage Diff              @@
##                dev    #7103       +/-   ##
=============================================
- Coverage     28.07%   10.90%   -17.18%     
=============================================
  Files          1288      673      -615     
  Lines        157729   113438    -44291     
  Branches       3121     1118     -2003     
=============================================
- Hits          44283    12368    -31915     
+ Misses       111569    99958    -11611     
+ Partials       1877     1112      -765     
Flag Coverage Δ
core ?
editoast ?
front 9.94% <ø> (ø)
gateway 2.52% <ø> (+0.18%) :arrow_up:
railjson_generator 87.49% <ø> (ø)
tests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Jun 17 '24 13:06 codecov-commenter

Tested Kubernetes driver against a real kubernetes cluster deployed for the tests. Worked properly.

ghost avatar Jul 14 '24 14:07 ghost

Since I started to work on this PR, we've revamped at multiple occasion the architecture of the scalable Async RPC. We ended up on something really different that what was first envisioned.

Closing this PR in favor of a one against with a new branch.

ghost avatar Jul 14 '24 20:07 ghost