Tracking subscriptions with devtools is not helpful enough

[alex-dixon]

The subscriptions we generate from rules are basically subscriptions to queries. Going forward they may play an even more prominent role in the framework than they do now. With this in mind, it seems valuable to be able to track subscription results using the devtools and interrogate the facts inserted by them about their origin.

Subscription results are inserted into the session as a map. The ORM tree (store/view-model) associates eid-of-sub -> sub/response -> key of sub map -> value. Because the map is inserted as the value of a fact, large subscriptions can be triggered as the result of a very tiny change. I've seen this here:

(defsub :products
  [[_ :products/list ?products]]
  [[_ :product-hovered ?e]]
  =>
  {:products ?products
   :hovered ?e})

When hovered changes from nil to an eid, the entire RHS of the rule fires again.

We might not expect Clara to be doing work in this case, when in fact the logical insert implicit in defsub retracts the previous subscription response then inserts the new one. In this case we've lost the microchange tracking that eav tuples should afford us: We see that the value of the sub response changed, but we'd need to manually diff it to find out that :hovered went from 1234 to nil is the only thing that's actually different. Our change tracking will only tell us (from state x to y):

state x
e :sub/response {:products <large amount of data> :hovered nil}
state y
e :sub/response {:products <large amount of data> :hovered 1234}

On the bright side, developing the devtools surfaces issues like these. Given our current constraints with subscriptions, we might notice this inefficiency (maybe by just looking at the full diff) and break this subscription up into two, such that the products key of the sub isn't causing an insertion and retraction every time. We're accurately tracking and logging the operations the rule engine performs. This is at least half of what we're after. The other is to show what information changed from one state to the next. From that standpoint, is noise, because it didn't change, even though the value of the overall subscription did.

We don't want to obfuscate the fact the rule engine is actually doing extra work here. Instead of integrating a diff for subscriptions in devtools, we should fix the problem. I will open another issue for this once I have some potential approaches. The approach should be consistent with the rest of our fact semantics, such that the facts subscriptions insert can be tracked and interrogated with the same level of granularly as we've come to expect normally.