structure
structure copied to clipboard
Increase "entities update since" performance
All stats for roughly 8.5k notes and 3k tags
General
-
leanTypeEnumFixer
has <10ms impact - The majority of DB-related time (>90%) is spent pulling data, not querying/getting a cursor. This was tested by running the query in
mongosh
with.explain("executionStats")
and.toArray()
.
First read
- The majority of the time is now spend outside
entitesUpdatedSince
(<2s out of 7–8s) - Notes DB read is 1–1.5s, tags 0.2–0.3s
Subsequent reads
- For subsequent calls the majority of the time is spent pulling note and tag data from the DB (60–70% and 20–30% of 0.8–1s respectively).
- The time to build the diffs is minimal (1–3% combined) – but only if the diffs are empty!
- If there is any difference, but it is small, the diffing time jumps by ~100ms (10% of total time extra).
- Full chaotic diffs (e.g. from forgetting to sort consistently) results in 5+ minutes of diffing.
- There is a consistent overhead around the actual (library provided) diffing of 1–3ms.
- Cache operations are not very time-intense (<5%)
- empty diff: read 1–3%, preprocessing <1%, write <1%
- minimal diff: read 1–3%, pre-processing 1–2%, write 1–2%
- ... (document other changes and learnings)
State pre-optimization (master)
Pristine read: 35–36s Empty diff subsequent read: 0.8–1s Minimal diff subsequent read: 1–1.2s
Stats "after optimization"
Pristine read: 7–8s Empty diff subsequent read: 0.7–1s Minimal diff subsequent read: 1.1–1.2s