fauna-js icon indicating copy to clipboard operation
fauna-js copied to clipboard

Add initial change feed support

Open ecooper opened this issue 1 year ago • 0 comments

DRAFT UNTIL THE README IS UPDATED.

This PR adds support for change feeds in the JS driver. To do this, we add a new ChangeFeedClient that can be initialized via Client.changeFeed(streamToken). ChangeFeedClient can then be used as a async generator for each page or event by event via flatten. nextPage is exposed for consumers who don't want to use the generator pattern.

const changeFeed = client.changeFeed(fql`MyCollection.all().toStream()`)

try {
  for await (const page of changeFeed) {
    for (const event of page.events) {
      // ... handle event
    }
  }
} catch (error) {
  // ... handle fatal error
};

Or via flatten:

const changeFeed = client.changeFeed(fql`MyCollection.all().toStream()`)

for await (const user of changeFeed.flatten()) {
  // do something with each event
}

Description

Generally, this PR has surfaced the underlying abstractions aren't really equipped to support more Fauna endpoints. There's a refactoring opportunity across clients and the httpclient, but those changes are likely backwards incompatible. So I tried my best to follow the existing conventions in the codebase.

  • ChangeFeedClient: Exposes change feed pagination via an async iterator via a public nextPage method. It also surfaces flatten to iterate through all events directly.
  • ChangeFeedClientConfiguration: The required configuration for the iterator. Primarily, optional start_ts and cursor.
  • ChangeFeedPage: Because there are errors that are embedded in events that are apparently fatal, ChangeFeedPage wraps the events wire response in an iterator. If it sees an event type of "error", it throws. If these aren't actually fatal, we can remove the iterator wrapper.
  • HTTPClient generics: HTTPClient interfaces have been updated to support generics. This is the first new API using request other than query.
  • wire-protocol.ts: Updated with the wire protocol as I understand it.
  • package.json: Updated to use the change feed env flag in docker calls. This will need to be removed when the image is updated.

Motivation and context

To support change feeds beta in the JS driver.

How was the change tested?

Unit, function, and integration tests were added for all new surface areas.

$ /Users/ericcooper/Workspaces/fauna-js/node_modules/.bin/jest --run-in-band ./__tests__/functional/change-feed-client-configuration.test.ts ./__tests__/functional/change-feed-client.test.ts ./__tests__/integration/change-feed.test.ts
 PASS  __tests__/integration/change-feed.test.ts
 PASS  __tests__/functional/change-feed-client.test.ts
 PASS  __tests__/functional/change-feed-client-configuration.test.ts

Test Suites: 3 passed, 3 total
Tests:       28 passed, 28 total
Snapshots:   0 total
Time:        2.147 s, estimated 3 s
Ran all test suites matching /.\/__tests__\/functional\/change-feed-client-configuration.test.ts|.\/__tests__\/functional\/change-feed-client.test.ts|.\/__tests__\/integration\/change-feed.test.ts/i.

Change types

    • [ ] Bug fix (non-breaking change that fixes an issue)
    • [x] New feature (non-breaking change that adds functionality)
    • [ ] Breaking change (backwards-incompatible fix or feature)

Checklist:

    • [x] My code follows the code style of this project.
    • [x] My change requires a change to Fauna documentation.
    • [ ] My change requires a change to the README, and I have updated it accordingly.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

ecooper avatar Oct 09 '24 17:10 ecooper