withObserver observeCount stays one behind if you set field after creation

[fivecar]

It seems that withObserver enhanced components don't observeCount correctly when you set a field shortly after creation, vs. setting the same field during creation. For instance, if you have a component like this:

const LiveCounter = compose<LiveProps, {}>(
  withDatabase,
  withObservables<{database: Database}, LiveProps>([], ({database}) => ({
    count: database
      .get<Post>('posts')
      .query(Q.where('location', Location.PORCH))
      .observeCount() as unknown as number,
  })),
)(Counter);

You'll get correct counts when you create a new Post this way:

await db!.write(async () => {
      return await db!.get<Post>('posts').create(post => {
        post.location = Location.PORCH;
      });
    });

But you won't get the right count when you create a new Post this way:

    const created = await db!.write(async () => {
      return await db!.get<Post>('posts').create(post => {
        post.location = Location.YARD;
      });
    });

    await created.updateLocation(Location.PORCH); // this is a @writer

In fact, oddly enough, you'll get the right count if you subsequently do the first type of creation again. I have a full-repro repo here. Run it in iOS simulator to see the results.

REPRO:

1. Click "Create Post w/Location" and observe count is now 1
2. Click "Create Post, Then Set Location," and observe count is still 1 (!)
3. Repeat step 1, and observe count is now 3 (!)

Interestingly, if you do a different Step 3, where you repeat Step 2 instead, you'll get a count of 2 -- which, though an improvement, is still not correct. If you follow that up with a Step 4 where you repeat Step 1, you'll get a count of 4 -- which is correct, but surprising, in that a single button press jumped two counts.

Full code here: https://github.com/fivecar/waterobserve. All interesting code in App.tsx.

[fivecar]

Ok - this is a throttle problem. That is, specifically, you get the expected behavior if you instead observeCount(false). So my near-term issue is solved.

I think, however, for the broader WatermelonDB community, there are three issues:

• The throttling never catches you up to the right state. That is, instead of setTimeout(250ms) after sending first count → gather all changes in those 250ms → send updated count if the count is different, the current code seems to send first count → ignore all changes in the next 250ms → do nothing further. I think the current implementation leads easily to incorrect code (i.e. I bet it would violate most people's expectations that throttling never catches you back up to reality).
• Relatedly, if we keep today's behavior, I'd recommend defaulting to observeCount(false). Because in that case, most people's code will remain functionally correct, and the people that need higher performance will then explicitly set a flag (with documentation that the throttle drops updates, instead of apportioning them out over time).
• Lastly (and this might just be my misunderstanding), the current behavior still violates my expectations because the throttle seems to be applied before my query filters. Since the query is based on Posts with Location.PORCH, it's weird to me that the throttle misses the second update, in that the series of events are thus: 1) I create and commit a new Post with Location.YARD (this record should not match the query criteria at all), 2) I then update the Post to Location.PORCH, which should be the first record that actually matches the query (and thus shouldn't be throttled). Am I misunderstanding?

LMK what you think.

I've really enjoyed learning about WatermelonDB so far. It's my first project on this platform, after many on Firestore and one on Realm Sync. Great work, everyone!

[janwirth]

I am having the same issue with the data being one step behind. This is an issue especially when binding watermelonDB values to input fields.

[janwirth]

I think this is related to #1513

[juansb827]

i am having the same issue, in my case is reproduced after doing something as simple as deleting 10 records at the same time.

I think something like this could be used as a workaround for cases when no throttling causes performance issues

  (query.observeCount(false)).pipe(throttleTime(250, undefined, { leading: true, trailing: true }));

We disable internal throttle and do it using the throttleTime operator.

[juansb827]

@fivecar the behavior you mention is the same as the default behavior for throttleTime operator , emit a value and ignore the rest of the values for the given duration (e.g 250 ms) .

So i guess the bug comes from the fact that they use throttleTime without trailing trailing: true

https://github.com/Nozbe/WatermelonDB/blob/master/src/observation/subscribeToCount/index.js#L26C5-L26C17 As seen in the comment of that line, they seem to be aware of this bug not sure why they haven't fixed that