v2 subscribe api

[jldec]

Parts 3 of https://github.com/opral/inlang-message-sdk/issues/70 - subscribe api which does not depend on solidjs.

• [x] RxJS Observable investigation
• [x] hand-written Observable (experiment)
• [ ] introduce RxJS subscribe reactivity for Store API
• [ ] changeset

[changeset-bot[bot]]

⚠️ No Changeset found

Latest commit: 5c0b01f495b9ce5c47810080d9724a3aecb41433

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[jldec]

RxJS Observable experiment and research

TL;DR handwritten Observable works, but it is hard to integrate with the rest of RxJS e.g. even to use basic operators like pipe, share etc. Proceed with dependency on rxjs v7.x for this PR.

• RxJS is the most widely used, 30k stars, mature library for Observables - current production version is 7.8.1 https://www.npmjs.com/package/rxjs

• primary contributor https://github.com/benlesh https://benlesh.com/

• decent docs https://rxjs.dev/guide/observable

• Trying to align v8 with WICG group observable proposal for improved browser event handing (NOTE the v7 api is not compatible with WICG observable) https://github.com/WICG/observable (nice readme, how it relates to promises api) https://wicg.github.io/observable/ (spec) https://github.com/ReactiveX/rxjs/discussions/7447

• v7 API has rough edges e.g. the need for pipe, and special handling of multicasting (ConnectableObservable, multicast, publish are all deprecated, share has surprises) https://github.com/ReactiveX/rxjs/discussions/7447#discussioncomment-8633441 (pipe or no pipe) https://rxjs.dev/deprecations/multicasting

• v7 is monolithic, v8 (future) will extract Observable https://github.com/ReactiveX/rxjs/issues/6786

• includes many operators, useful but not all relevant for basic (sdk) library api https://rxjs.dev/guide/operators#categories-of-operators

• decent TypeScript support https://github.com/ReactiveX/rxjs/blob/7.x/src/internal/types.ts https://github.com/ReactiveX/rxjs/blob/master/packages/observable/src/types.ts (v8-alpha)

• not moving forward very actively https://github.com/ReactiveX/rxjs/graphs/contributors?from=2023-04-01&to=2024-06-08 https://github.com/ReactiveX/rxjs/discussions/7447

• TC39 observable proposal stalled https://github.com/tc39/proposal-observable

• James Henry (NX) extracted Observable package but nothing since then https://github.com/ReactiveX/rxjs/commits?author=JamesHenry

[samuelstroschein]

@jldec awesome write up, thank you. I deem that you, @martin-lysk and I are aligned that RxJS observables is the basis for *.subscribe()? reasoning:

• only major (?) production reactive system that is not framework-bound
• adapters to framework reactivity is trivial
• standardization processes around observables and signals are still years away (sad)

cc @janfjohannes who has experience with rxjs

[janfjohannes]

@jldec the important missing info is why hand written observable is not playing nice with the operators. what exactly are the issues?
as i said i recoend adopting minimal surface area, if we get away with minimal internal subscribable totally fine. especially in the early days when rxjs 4 was quite slow companies like netflix used a minimal internal obeservable implementation with support for external operators and interop, so this was a common practice leading mostly to issues as the spec moved and there were incompatible apis leading to them using apis prefixed with classicObservable. as recent rxjs has much better performance and does not bundle all operators immediately the need for this pattern was reduced, but can still make sense if it works and is compatible enough to be able to replace without api change.

[jldec]

@janfjohannes - you were right, with a little bit of extra work (subscribed state management) and the introduction of a @@observable symbol, and the use of rxjs.from(), the hand-written observable does play nicely with the RxJS operators like pipe and share.

I can maybe simplify things even more using from(asyncIterator), but here is the native RxJS watchFs observable and here is the hand-written watchFs observable.

Example hand-written watcher usage from the test here

const watcher = watchFs({ nodeishFs, baseDir }) as unknown as rxjs.ObservableInput<string>

const unshared = rxjs.from(watcher)
const shared = rxjs.from(watcher).pipe(rxjs.share())

the equivalent with a native RxJS observable watcher looks like this

const watcher = watchFs({ nodeishFs, baseDir })

const unshared = watcher
const shared = watcher.pipe(share())

I'm going to move forward with the introduction of the RxJS dependency in this PR while we continue our evaluation of RxDB.

[samuelstroschein]

closing because outdated