Optimize session list loading by stripping diff content
Problem
Session list loading takes 10+ seconds when sessions contain large diffs (hundreds of MBs). The /session endpoint returns full summary.diffs with before and after file contents, but the list view only needs file names and change counts.
Solution
Strip before/after fields from diffs in Session.list(), keeping only metadata (file, additions, deletions). Made FileDiff.before/after optional to support both lightweight (list view) and full diffs (message details).
Added session-loader.ts to handle batched reading (10 concurrent) and optimize large file parsing (>100KB reads only first chunk).
Hi @rekram1-node, it's me again! It's great if you help me review this PR. Thank you 😄
thanks for pr @yudgnahk will review when I get a chance
Plugging in a tangent issue
- #3526
@zenyr you still got the issue with the latest version?
i think dax did some optimizations for this could be wrong, how is it working for u on latest
@zenyr you still got the issue with the latest version? i think dax did some optimizations for this could be wrong, how is it working for u on latest
Sorry for the late reply, totally missed out the mention. My bad!
- Unfortunately I wiped out 3GBs of session data after that report...
- However, accessing the session list feels much snappier than before.
- Yet, jumping into the session is still a bit slow, combined with occasional rendering failure(when a single message significantly overflow the screen? Not sure.)/messages coming in one-by-one-ish.
- Having said that, I think it's overall better now.
... That's a lot of 'buuut'. As always thank you for the great project.