re-frame-10x
re-frame-10x copied to clipboard
Rebuild Epoch Parsing
The epoch parsing code is in need of significant refactoring and bug fixes. It has led to the following issues:
- delays in rendering content: #264
- lagging event history: #268
-
dispatch-sync
invisible: #154 -
nil
app-db state: #263, #151 - inconsistent use of event filters: #175
- duplication of subs between the current epoch and intra-epoch subs
- invalid timing information
The relevant code is in ::receive-new-traces
, parse-traces
, surrounding code in the metamorphic
ns and in subs that use the app-db state derived from the epoch parsing.
Step 1: Move ignored-events
filtering from event to subs
In ::recieve-new-traces
there is a remove
to filter ignored events (via settings) before epochs are stored in app-db.
This causes some issues:
- Partial epoch states in app-db leading to odd workarounds in subscriptions which are not all handled
- If the user removes or adds an epoch filter with existing traces in app-db, 10x is in an invalid state as it does not have a correct 'view' of epochs according to the settings.
The remove
filter needs to be deleted from ::recieve-new-traces
and added to the appropriate subs instead.
Step 2: Change metamorphic/parse-traces
so that dispatch-sync
is its own epoch
The delimitation of dispatch-sync
epochs is not detected in metamorphic/parse-traces
. This needs to be fixed so a dispatch-sync
is returned as a distinct epoch.
Step 3: Move all parsing from ::recieve-new-traces
event to metamorphic
ns
::receive-new-traces
contains too much data transformation/parsing related code. If this is moved to metamorphic
ns it would be easier to write unit tests to debug/check correct behavior and so forth.
New APIs/functions should be considered as part of this move, as should renaming. The current names are certainly not great.
Tests should be added for all the significant epoch parsing use cases/situations.
At least the following should be moved:
-
parse-traces
-
parse-state
changes -
subscription-info
-
subscription-match-state
-
subscription-matches
-
new-sub-state
-
timing
mapv -
new-matches
map -
quiescent?
-
all-matches
-
retained-matches
-
first-id-to-retain
-
retained-traces
-
:matches-by-id
into...map -
:selected-epoch-id
-
:selected-epoch-index
Step 4: Performance
Epoch parsing and associated code is currently the biggest hit on 10x performance.
Once more of the code is moved into functions in metamorphic
with unit tests various optimisations could be explored such as storing the internal parse state as transient vs persistent collections, doing less string parsing etc.