Epic: Tracking accuracy

[ErikBjare]

This issue is meant to track accuracy issues present in ActivityWatch.

Inaccuracies can arise in two places: during collection, and during analysis.

Collection

Bugs in how events are sent, queued, and merged can affect the data and lead to inconsistencies like unexpected gaps and overlaps.

Inaccuracies introduced by bugs during collection are taken very seriously, as they may lead to unrecoverable loss of usable tracking data. However, spurious behavior and difficulty in testing well presents difficulties in ensuring things are bug-free.

Some inaccuracies arise from assumptions made during collection, such as setting constants for polling intervals, AFK timeouts, etc. These should be intelligently set to minimize inaccuracies and make a good tradeoff between data detail and space/compute requirements.

Known issues

• There are occurrences where events are duplicated, often then keep getting heartbeats such that there are two events with the same start and data but different durations. See here: https://github.com/ActivityWatch/activitywatch/issues/239#issuecomment-821085836
  • May downstream lead to warnings about overlapping events and negative gaps.
• Sometimes, there are weird occurrences of very long-stretching events passing multiple others, such as in this screenshot: https://github.com/ActivityWatch/activitywatch/issues/602#issuecomment-970240728
  • Cause unknown

Analysis

Bugs in transforms and queries may lead to analysis issues. However, these bugs are generally considered less critical since as long as the underlying data is correct, the analysis can always be corrected later. However, they can still present significant frustration for end-users, who may find the resulting buggy analysis results useless.

Transforms and queries are also easy to test, and the dual-implementation of transforms in both Python and Rust make it a suitable target for cross-implementation testing to ensure robust analysis methods.

Known issues

• #239
• #602
• #626
• #722