Dynamic capsules side effect ("families")

[GregoryConrad]

Useful for dynamic programming, incremental computation, recursive capsules, and weird one offs.

Update: for how to use, see https://github.com/GregoryConrad/rearch-dart/issues/221#issuecomment-2380717712

TODOs

• [ ] (MAYBE) SideEffectRegistrar.cleanup, which would be used to cleanup resources without causing non-idempotence.
  • [ ] Could be used by the dynamic capsules side effect to remove capsules from the underlying DynamicOrchestrator's map when they are disposed.
  • [ ] cleanup could also be used for widget updates--we could remove onNextUpdate with widgets creating a dummy capsule themselves.
• [ ] Provide enhanced idempotent GC to onNextUpdate to prevent any leaky behavior when dynamic capsules are read from widgets.
• [ ] Adequate testing (to ensure that things like dependency capsules being updated don't break a set of dynamic capsules, for example)

[TekExplorer]

Ref: riverpod families

[GregoryConrad]

Thinking an API something like:

Family<Foo, Bar> fooBarFamily(CapsuleHandle use) {
  return use.family((use, Foo foo) {
    final client = use(httpClientCapsule);
    return Bar(client, foo);
  });
}

@rearchWidget
Widget example(WidgetHandle use, Foo foo) {
  // The below syntax requires some fun trickery, see next code snippet for impl
  final Bar bar = use(fooBarFamily[foo]);
  return BarWidget(bar: bar);
}

To support the use(fooBarFamily[foo]) syntax, we can do a cool:

extension FamilyAccess<Param, Return> on Capsule<Family<Param, Return>> {
  Capsule<Return> operator [](Param p) => (CapsuleReader use) => use(use(this)[p]);
}

[busslina]

Can you explain this side effect? The Riverpod one refers to identified instances of a kind (type), but here having two generic types confuses me.

Also, does the callback passed to use.family executes synchronously?

The extension seems to answer my self: yes.

[busslina]

Can you explain this side effect? The Riverpod one refers to identified instances of a kind (type), but here having two generic types confuses me.

I think I understood... Foo is the key.

[GregoryConrad]

It is akin to Riverpod families for really specific scenarios, or for odd one offs that can't be solved effectively with factories. They will not be recommended for normal use and are strictly reserved for advanced use cases.

Not sure I'll call the resulting solution "family", since it's not too intuitive unless you're coming from Riverpod, but that's the placeholder name for now.

Also, does the callback passed to use.family executes synchronously?

The family side effect will essentially just create capsules for each parameter passed in (think a Map<Param, Capsule>). That callback will provide the body of each newly created capsule.

Can you explain this side effect?

In the possible API i gave above, Foo is the family key; Bar is the return value.

[GregoryConrad]

In the current experimental implementation, dynamic capsules (I'm calling "families" that to be consistent with the Rust implementation/docs) would keep all of the keys/variants stored in the Map even when the data of the dynamic capsules themselves are destroyed. I'm not really sure this even constitutes as a problem, but if it were to be "fixed," one of the following would need to happen:

abstract interface class SideEffectApi {
  void forceIdempotent(); // new method that tells ReArch to disregard this particular side effect registration
}

abstract interface class SideEffectRegistrar {
  // register(), as before
  /// Is called whenever this capsule changes, whether that means it is rebuilt or disposed.
  /// Can be used to cleanup a resource.
  void cleanup(void Function()); // this f
}

Or maybe something similar. Not sure what I like more at the moment.

[GregoryConrad]

Update: the following has been added under the top-level experimental.dart file

May also add some helper functions to experimental.dart to make stuff more ergonomic:

final myCapsule = capsule((use) {
  return use.data(123);
});

final myDynamicCapsule = dynamicCapsule((use, Param param) {
  return use.data(param);
});

// ...
use(myCapsule);
use(myDynamicCapsule[Param(123)]);

Not entirely sold on them since they:

1. lack type information in source code
2. are not const, but rather final since closures can't be const yet

But I think this would make sense to include and users can opt-in when they want.

[GregoryConrad]

For those following, here's a test case that shows how to use this new (experimental) side effect:

  test('fibonacci numbers', () {
    DynamicCapsule<int, BigInt> fibonacciCapsule(CapsuleHandle use) {
      return use.dynamic((use, n) {
        return switch (n) {
          _ when n < 0 => throw ArgumentError.value(n),
          0 => BigInt.zero,
          1 => BigInt.one,
          _ => use(fibonacciCapsule[n - 1]) + use(fibonacciCapsule[n - 2]),
        };
      });
    }

    final container = useContainer();
    expect(
      container.read(fibonacciCapsule[100]).toString(),
      equals('354224848179261915075'),
    );
  });

I'll publish a new release shortly with the first iteration of the side effect, but keep in mind things may leak/not work perfectly for the time being.

[GregoryConrad]

New version released (1.12.0) with experimental dynamic capsule implementation. It seems to work, but I wouldn't recommend using it for anything mission-critical anytime soon.

1.12.0 also adds a new experimental syntax to create capsules:

final countPlusOneCapsule = capsule((use) => use(countCapsule) + 1);

final countPlusDynamicCapsule = dynamicCapsule((use, int i) => use(countCapsule) + i);

// ...
final countPlus123 = use(countPlusDynamicCapsule[123]);

[TekExplorer]

dynamicCapsule

Okay, but hear me out: capsule.dynamic((use, arg) => ...)

[GregoryConrad]

I've thought about this some more and decided on a few things:

• forceIdempotent, as mentioned here, is not going to happen. It clashes far too much with the rest of ReArch's design.
• I could add a cleanup style method on SideEffectRegistrar. The issue is that I never marked it @experimental, so it's possible someone using the library in some really weird way could break. That might, however, be a risk worth taking.
  • Instead of cleanup on the SideEffectRegistrar, I could instead add it on SideEffectApi, but that would present numerous other problems. So that won't likely happen either.
• I vetted all the code again today (didn't take that long since the entire core implementation is only a few hundred lines of code--one selling point of ReArch 😄) and realized there wouldn't be any problems regarding dynamic capsules--everything is properly handled as-is. The only things is that onNextUpdate with dynamic capsules may be a bit leaky unless we:
  • (1) Trigger idempotent garbage collection a few levels deeper (say 5 or so), or
  • (2) Trigger idempotent garbage collection all the way until the "root" nodes of any branches used to build the leaf
  • (I'd probably just go with the first option to save implementation time & complexity, and do less at runtime)
• I think I'd want to rename some of the types:
  • typedef DynamicCapsule<Param, Return> = Capsule<DynamicOrchestrator<Param, Return>>;, or something like that.
  • This way we can refer to dynamic capsules as DynamicCapsules directly.
  • And users won't try to treat dynamic capsules as regular capsules, because even though they are, they are a bit useless as such and require that special [] syntax to read out of them
• I do like capsule.dynamic((use, arg) => ...), but I'm not entirely sold on it yet
  • Mostly because capsule() would mean capsule would be a top-level variable with a call(), which would hurt the IDE experience just a little bit. Alas, here's some code you can use in your own project, since capsule and dynamicCapsule are still experimental anyways:

class _CapsuleCreationImpl {
  const _CapsuleCreationImpl();
  Capsule<T> call<T>(Capsule<T> cap) => cap;
  Capsule<DynamicCapsule<Param, Return>> dynamic<Param, Return>(
    Return Function(CapsuleHandle, Param) dyn,
  ) =>
      dynamicCapsule(dyn);
}

const capsule = _CapsuleCreationImpl();

// Usage:
final cap = capsule((use) => 123);
final cap2 = capsule.dynamic((use, int i) => '$i');

Edit:

On second thought, one thing to note about the capsule.dynamic sort of deal is that we can create shorthands for all sorts of different side effects through it, like:

final Capsule<(String, void Function(String)> statefulStringCapsule = capsule.state('');
final Capsule<ValueWrapper<int>> intDataCapsule = capsule.data(123);
// etc.

And once you start to build out a full app with ReArch, like I just did with examples/hevy_smolov_jr, is that defining the same sort of capsules over and over can get old. And users can define their own extensions this way too. So I'll give this some more thought. Only concerns are:

1. IDE experience (as mentioned above)
2. New users may get confused at all the different ways to define capsules