dart-lang
Enable functions to store data between calls / know from where they were called
Hi,
this title might sound quite strange but during the development of functional_listeners and get_it_mixin I found that the reactive nature of flutter creates some new challenges. Specifically I mean that build functions are getting called on any data change which makes patterns that I used in the past suddenly difficult.
Let's take the following class as an example:
class Model {
ValueNotifier<String> _notifier;
ValueListenable<String> get nonEmpty => _notifier.where((x) => x.isNotEmpty);
ValueListenable nonEmptySafe;
Model() {
nonEmptySafe = _notifier.where((x) => x.isNotEmpty);
}
Future<String> makeRestCall() {
return Future.value('Result from ');
}
}
Defining a getter like nonEmpty is no problem in classic UI systems that will only call it ones from one place in the code.
where creates a new special ValueNotifier that the caller can subscribe to.
If we access this inside a widget like:
ValueListenableBuilder<String>(
valueListenable: widget.model.nonEmpty,
builder: (context, s,_) {
it works fine the first time the build function is called. But if the surrounding context gets rebuild it will lead to a new instance of that ValueNotifier is created while the first one still exists and reacts on data changes of _notifier.
the only way to prevent this is to assign the result of where to a normal field or variable outside the build function like it is done with nonEmptySafe.
Because of the same reason it's not a good idea to call a data tranformer like whereof functional_listeners or rxdart directly in a constructor of a widget like:
ValueListenableBuilder<String>(
valueListenable: widget.model.nonEmpty.map((x)=>x.toUpper),
builder: (context, s,_) {
another place where you see mistakes like that is in combination with FutureBuilder that gets it's future from a method call into the model layer. If the context rebuilds suddenly the Futurebuilder wait for a new Future and not for the one it first got. Like if you call makeRestCall() in the model above. what you really wanted is that the FutureBuilder will wait for the first call and not that it does a new call because someone higher up the widget tree did a setState.
The problem here is that makeRestCall() cannot do anything against this because it doesn't know that it is called from the excact same place again.
To solve this we would need a possibility that a function is able to store data at the place where it is called that will persist between consecutive calls or that a function could query the adress from where it is called so that it can differentiate different calls and can store data in a static map for instance.
By this where could only create a new instance of the ValueNotifier that it has to return on the very first call and just return the same instance on all successive calls.
I could imagine several approaches
- add a
callIdkeyword that returns a unique value for every call position in the code. (this could be just the current return address on the stack) - add a new type of variable
static localthat keep their value from one call at a calling position to the next - add a special keyword
localStorage<T>that allows to store one object between calls
(option 1 is my least favorite because it would lead to static or global maps to reall store the data)
Especially extension functions could profit of such a feature as they have no way to add a new field to their target types if they needed to store any data.
Another example that would benefit from this is that current "hacks" like flutter_hooks or my get_it_mixin could be implemented in a clean way. Currently we increment an index everytime one of its functions is called that is reset to 0 at the beginning of the build function so that we can identify which call position we are at and to store necessary data. this also means that this calls always have to be done in the same sequence and can't have any conditionals in it.
With the proposed feature this wouldn't be a problem anymore because every function call could be completely independ of other.
I'm aware that such a feature should only used were no other option exist, but especially package authors could add safety messures to their functions that make them save to call inside build functions.
Good idea. But it has some "history". Discussed on several occasions and quickly forgotten. IMO, the most realistic way for it to be added is by supporting"static-local" variables in any place of the code.
if (something) { static final id=Object(); foo(x, y, z, id); }You still have to pass id explicitly, but it's not a very big deal. But if you want to ever finish your project, please don't wait until this feature is implemented - it may take a while :-) Just declare a top-level
static final _callSiteId1 = Object()in your code and pass it. You can have as many ids as you want (one per call site).
How would that help me to automatically get the correct Id when my function is called the next time from the same place?
My packages are alerady published but without this feature users will have to take extra precaution when using them in a build method.
Ah you mean I have to pass the ids from outside into the function, but exactly that I want to avoid. The user of the function shouldn't know anything about this
I think 7 years ago there wasn't a reactive framework that would benefit of it.
A loop wouldn't be a problem as it would be the same call site, it's actually what this is for in a sense, that I can detect that I'm called multiple times from the same place. If it gets called from two different places then is should be treated as different calls.
I think we have features sufficient to make this work already. Yes, creating a new listener every time you ask for one is probably a problem. Not an insurmountable one, if no-one listens to them, they shouldn't matter, but there is the risk of them never being garbage collected and wasting space.
If you really only need one nonEmpty listenable, then only create one, as you recognized. There are sevearl ways to achieve this:
late final ValueListenable<String> nonEmpty = _notifier.where((x) => x.isNotEmpty);
or
ValueListenable<String>? _nonEmpty;
ValueListenable<String> get nonEmpty => _nonEmpty ??= _notifier.where((x) => x.isNotEmpty);
or even
static final Expando<ValueListenable<String>> _nonEmpty;
ValueListenable<String> get nonEmpty => _nonEmpty[_notifier] ??= _notifier.where((x) => x.isNotEmpty);
The first two are idiomatic ways to initialize a variable only once. I think the late final is particularly short and readable.
(I notice that _notifier is mutable and unintialized in your example. I assume it's final and intialized in the real code, otherwise I have more worries! I chose to use _notifier instead of this in the Expando example because it works with a mutable _notifier too).
There have been talks about a static expression or variable declaration which ensures that an expression is evaluated only once, ever, without having to declare a lazy static variable for it. Something like either:
foo() {
static var x = init; // initialized only once, every time after that it uses the first value.
... or ... use(static expr); // also evaluates expr only once, then uses the same value every time after the first.
}
Those are basically equivalent to anonymous static variables initialized the first time the declaration/expression is evaluated.
(I prefer the static expr because then the implicit variable can be made final, which makes it much easier to reason about. I don't know what it would mean if someone assigned to x above).
What you are asking for here is something similar, but for instance variables. We don't have a good name for instance variable, so let's strawman and use class. Something like:
ValueListenable<String> get nonEmpty => class _notifier.where((x) => x.isNotEmpty);
would be equivalent to having a variable which is initialized the first time this is evaluated, and then read every later time. (So basically
ValueListenable<String>? _nonEmpty;
ValueListenable<String> get nonEmpty => _nonEmpty ??= _notifier.where((x) => x.isNotEmpty);
except that we have to be more clever if the type is nullable).
I can see uses for this, for caches mainly.
But to be honest, I'd prefer linking the cache to the _notifier object. If your _notifier is mutable and is changed, it would be great to not retain the old cached nonEmpty listenable.
It won't work on classes with const constructors (or it could, if we implement it using Expandos instead of real fields).
It can also cause unexpected memory bloat if a commonly used class suddenly gets several extra fields. Probably not a real issue.
@lrhn I think you missunderstood me. I know that you can prevent that problems. But I as a package author want to prevent that they can happen at all, so that users of my functions don't have to deal with this. It also doesn't help with my other apllication to implement Hooks in a really save manor.
I would argue the other way round, because it's specific to Flutter it makes sense to support it by Dart as it's one ecosystem. I give it a try. In Flutter everything is a Widget. Widgets all have a build method that is called to update the UI. As it is a reactive framework, changes in your data are updated in the UI by rebuilding the Widgettree that contains the UI part which underlying data has changed.
To notifiy the UI that it has to rebuild a part of the app there are different approaches. One is to use a ValueListenableBuilder Widget that rebuilds whenever the value of the provided valueListenable changes. In the following example the ValueListenable is accessed by using the ServiceLocator get_it
class TestStateLessWidget1 extends StatelessWidget {
@override
Widget build(BuildContext context) {
return Column(
children: [
ValueListenableBuilder<String>(
valueListenable: GetIt.I<ValueListenable<String>>(instanceName: 'currentUserName'),
builder: (context, val,_) {
return Text(val);
}
),
],
);
}
}
As such builder widgets clutter the UI tree with update logic making it harder to see what will finally displayed there have been alternatives developed, namely flutter_hooks and get_it_mixn. I will use my mixin in the following as its a bit easier to show, but as it is internally similar implemented like Hooks the consequences apply to both.
With the mixin you can write the above like this:
class TestStateLessWidget1 extends StatelessWidget with GetItMixin {
@override
Widget build(BuildContext context) {
final currentUser =
watch<ValueListenable<String>, String>(instanceName: 'currentUserName');
return Column(
children: [
Text(currentUser)
],
);
}
}
Which is much easier to read. watch returns the current value of the ValueListenable<String> with name 'currentUserName' that is stored inside the service locator. But besides that it registers a listener under the hood that will trigger a rebuild of this widget whenever that object changes.
Now we already have the problem that only the first call of watch must register this handler, every following call should not register another handler but only return the current value.
As all build functions have to expect to be called multiple times by the framework watch needs to store the information if its the first or a following call somewhere. This is also necessary to be able to unregister the handler automatically when the Widget gets disposed.
Now imagine that we don't have just one but multiple watch statements at the beginning of a build function:
class TestStateLessWidget1 extends StatelessWidget with GetItMixin {
@override
Widget build(BuildContext context) {
final currentUser =
watch<ValueListenable<String>, String>(instanceName: 'currentUserName');
final currentUserEmail =
watch<ValueListenable<String>, String>(instanceName: 'currentUserEmail');
return Column(
children: [
Text(currentUser),
Text(currentUserEmail),
],
);
}
}
So this will rebuild this widget whenever either of the two ValueListenables change. How can watch keep track on this now? It's actually a quire ugly hack that was invented by hooks. The information is stored in a List that gets a new entry for every first call of watch inside this Widget. On every following call for every watch call an index is incremented to acess the correct information for the specific call.
This has the unpleasant consequences: The sequence for every call of build has to stay exactly the same. so no conditionals are allowed.
With the proposed feature this could be avoided as watch could internally distinguish one call from another or depending of the implementation it could store a different information for every call location.
I have a lot of concerns about what this would look like. It's possible that this could be a use of macros, given an appropriate macro system. cc @jakemac53 @munificent . You'd need to be able to add a piece of global state with a macro, and then not canonicalize the macro so that each invocation of the macro (corresponding to a call site) would end up as a distinct invocation. This feels like asking for memory leaks though. How do I GC the state associated with a call site? And if I don't.... then my program basically ends up with a persistent global for every relevant static call site. Maybe that's ok? Not sure.
Ya reading this before I even got to your comment @leafpetersen I immediately thought macros. I think the problem you describe around memory concerns are valid, but would not be specific to macros (all proposed solutions here likely would involve some permanent state per call site).
This also made me think of the flutter widget transformer which injects extra arguments into constructor invocations which tell it where the invocation came from. If we could turn that into a macro, that would probably be a large win as that kernel transform is a large source of technical debt.
I will keep the use case in mind.
1. how does `watch` know what widget it belongs to? While calling "watch", you never pass "this" as a parameter. Then how does your framework know which object (or objects) have to be rebuilt when the value changes?
:-) you see, it's a mixin? The mixin overrides createElement of the Widget class and smuggles his own implementation of element into the tree. watch is a member of the mixin.
2. why does `watch` have 2 type parameters? The second one is clear: it's the type of the returned value. But the first one... if it's always `ValueListenable<T>` where `T` is the second type parameter, it seems unnecessary. Please explain.
Sorry for that confusion. That's how the service locator get_it that I use here works. Actually the instance name is optional. It locates the registered Object by this first generic type parameter. In a real app you wouldn't register Valuelistenable<String> but some MyModel object and use the watchX method.
3. what if your widget depends not on one, not on two, but on N (variable number) listenable values? E.g. on every value of some list? Then you have to `watch` a variable number of these values, which requires a kind of loop. How will your program look in this case? (Maybe you have a special arrangement to cover this possibility? Like `watchAny` or something?)
Actually this wouldn't be possible to solve at this moment. Indeed a watchAny could solve that but in typical Flutter Apps you would solve that differently. If you would have to react to that many Objects you would group them into a class that Implements ChangeNotifier and call notifiyListeners for any field change.
The problem with using the return address is that Dart code might live on the heap, and it can be moved by garbage collection.
That's not a complete blocker for the idea, as long as the CallerReference object is opaque and can be updated by the same garbage collection to point to the new place (and as long as it won't get into an inconsistent state during incremental GC), then it can still work.
Another issue is that the same code can be recompiled and JIT-optimized, and there might be more than one version of the code at the same time, unless we do on-stack replacement and migrate everything at the same time. (We probably do).
The biggest issue is that I can't see how to compile it to JavaScript. The arguments.caller caller reference is disabled in all modern JavaScript (if using "use strict", the function won't allow itself to be seen in arguments.callee). That was a feature explicitly introduced to do what is being asked for here, and it was dropped again because it wasn't a good feature in practice. That makes it unlikely that we'll get a replacement that can be used in JavaScript.
I haven't forgotten this but I need some quiet minutes to think this through again. Thanks for all the replies so far.
@escamoteur, can you be a bit more general about the use-case? From your example all the way up top, I also immediately thought of late final or using ??=. But as you pointed out, using a regular field constructed in the initializer sounds like the way to go. A getter is saying "run this code every time this value is queried". If that's not what you want, you should use a regular field. Am I missing the bigger picture?
With my package watch_it getting more popular and the wish as @mit-mit expressed it in the recent "Flutter in Production" event to make Flutter widget trees more streamlined, I think this issue should be revisited. Not sure where we are with macros now, if that might offer some solutions.
Still the very nature of constantly rebuilding widgets would benefit from a solution to this problem. Hook-like state management and resource management libraries allow to write way better readable Widget trees.
Could you explain how any of the proposed solutions can help, unless the user explicitly passes some callId? The fact that two calls come from the same line doesn't mean they are related in any way. E.g. you might need to create 100 similar widgets - so you write a single function that generates these widgets with different parameters (e.g., for one, "label" parameter is "Alice", for another it's "Bob", etc). They all get generated by the same line.
I'm not very knowledgeable about flutter, but... isn't Key parameter supposed to uniquely identify the widget?
OK, just to summarize the advatanges of an reliable id of the calling site of a function, ideally the position in the source code, or some abstract ID that can then be converted into a real source code location with file name and line number
- all hook based state management solutions
flutter_hooks,riverpod_hookandwatch_itwould benefit because user could call the hook functions anywhere inside a build function safely without worrying about conditionals or loops - tracing and error reporting could be implemented more easily without extracting the current stack trace and analysing it
For the first one the current return address from the stack would be completely enough and either with a #pragma we could mark such function from not to be inlined when AOT or maybe the compile is smart enough to see that we access the return address via some language feature so it will never inline functions using it. It's probably the easier solution that introducing an abstrract source location which definitely would be nice
This is an old problem that's been around as long as the Observer pattern has existed. I remember dealing with it in C# WinForms apps in the early 2000s.
If you have some kind of observable object that maintains a list of listeners, then you need to be mindful of un-reginstering listeners that you no longer need. Otherwise you get zombie objects. Really, you need to think of any object that implements the observer pattern as being a resource whose lifetime matters. Registering a listener is sort of like opening a file handle, and if you never unregister it, the handle stays open.
As you note, the problem is here:
ValueListenable<String> get nonEmpty => _notifier.where((x) => x.isNotEmpty);
That where() call creates a new ValueListenable and registers it as a listener on _notifier, but that listener never gets unregistered. The program may correctly unregister the widgets that are listening to nonEmpty, but the middleware listener created by each access of nonEmpty never gets unregistered from _notifier.
One API-level solution might be to have smarter adapter listeners. When the last listener is unregistered from a middleware listener like nonEmpty here, it automatically unregisters itself from its upstream data source. That way it's no longer still being sent notifications that it will drop on the floor anyway.
I could be convinced, but I don't think a solution based on knowing where function calls appear syntactically would be a big help here. It doesn't look like it would help at all in this example. If you had:
class Model {
ValueNotifier<String> _notifier;
ValueListenable<String> get nonEmpty {
static listenable = _notifier.where((x) => x.isNotEmpty);
return listenable;
}
ValueListenable nonEmptySafe;
Model() {
nonEmptySafe = _notifier.where((x) => x.isNotEmpty);
}
Future<String> makeRestCall() {
return Future.value('Result from ');
}
}
Then, yes, you wouldn't create a new listenable every time nonEmpty is called. But you wouldn't create a new one even if nonEmpty was called on different instances of Model. If you have two different model objects, presumably they really should their own event streams!
Actually the ValueListenable chain is the less important reason for this. And still knowing the calling location would help as the extension method could look up in a map if it has already created that object.
But the important application is hook like state management in Flutter apps. And there is would absolutely help and remove a limitation of breaking of you use any conditional.
Am 6. Nov. 2025, 18:36 -0500 schrieb Bob Nystrom @.***>:
munificent left a comment (dart-lang/language#1427) This is an old problem that's been around as long as the Observer pattern has existed. I remember dealing with it in C# WinForms apps in the early 2000s. If you have some kind of observable object that maintains a list of listeners, then you need to be mindful of un-reginstering listeners that you no longer need. Otherwise you get zombie objects. Really, you need to think of any object that implements the observer pattern as being a resource whose lifetime matters. Registering a listener is sort of like opening a file handle, and if you never unregister it, the handle stays open. As you note, the problem is here: ValueListenable<String> get nonEmpty => _notifier.where((x) => x.isNotEmpty); That where() call creates a new ValueListenable and registers it as a listener on _notifier, but that listener never gets unregistered. The program may correctly unregister the widgets that are listening to nonEmpty, but the middleware listener created by each access of nonEmpty never gets unregistered from _notifier. One API-level solution might be to have smarter adapter listeners. When the last listener is unregistered from a middleware listener like nonEmpty here, it automatically unregisters itself from its upstream data source. That way it's no longer still being sent notifications that it will drop on the floor anyway. I could be convinced, but I don't think a solution based on knowing where function calls appear syntactically would be a big help here. It doesn't look like it would help at all in this example. If you had: class Model { ValueNotifier<String> _notifier;
ValueListenable<String> get nonEmpty { static listenable = _notifier.where((x) => x.isNotEmpty); return listenable; }
ValueListenable nonEmptySafe;
Model() { nonEmptySafe = _notifier.where((x) => x.isNotEmpty); }
Future<String> makeRestCall() { return Future.value('Result from '); } } Then, yes, you wouldn't create a new listenable every time nonEmpty is called. But you wouldn't create a new one even if nonEmpty was called on different instances of Model. If you have two different model objects, presumably they really should their own event streams! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>
TBH I am not equipped to follow all the execution details, but I AM equipped to talk about the pain of not being able to have conditional hooks. Betterment uses hooks as a core state management component. To get around them not being able to be called conditionally, we are in the practice of adding skip parameters to our hooks that, when true, internally no-op themselves and/or breaking each conditional path into helper widgets containing the optional hooks that themselves are conditionally rendered. While both of these work, they're very complicated and their intent is non-obvious. They especially trip up junior devs and have been a constant source of bugged code that we (at least thus far) have relied on dogfooding and code review to keep from turning into production incidents.
Here's an example of a bugged hook that we caught in dogfooding and had to panic fix right before a launch (just one example, but we have many foot guns like this in our codebase, misusing hooks that should've been conditional is probably our biggest single source of bugs rn):
Widget build(BuildContext context) {
// A hook we use to interact with our GQL client.
// Returns `(Foo?, Exception?)` where foo is non-null iff successfully fetched.
// Triggers rebuilds when state changes, including when client issues a mutation-`useQuery`
// retriggers with the new data.
final (hasOptedIn, error) = useQuerySelector(userOptInState, (data) => data?.user.hasOptedIn);?
// Intended to only run once after the data fetch.
useEffect(() {
if (hasOptedIn == null || !hasOptedIn) return;
// The user has already opted in! Skip to the next step.
context.flow().setStep(SomeOtherStep());
// When any key in this list changes, effect re-runs.
// We need this to ensure the hook runs post-load.
}, [hasOptedIn]); // <== HERE IS THE BUG!
// Other logic here including more hooks calls that were snipped for brevity...
if (hasOptedIn == null) return LoadingScreen();
return OptInScreenContent();
}
So the bugged case is really subtle. We can't run the useEffect hook conditionally so we re-run the effect whenever hasOptedIn changed. This is a common pattern in hooks-land that a dev will instinctively reach for. BUT! there's a bug: since we just used the bool?directly, when the user DOES opt in, it'll change from false->true and retrigger the effect! So the user will be skipped to a later step in the middle of a UI interaction. On some level, the dev who wrote the bug (me, but don't tell anyone ;) ) should just get good and use hooks better:
useEffect(() {
...
}, [hasOptedIn == null]); // <== Now only runs once on first load
...but it's much easier to rely on APIs that aren't foot guns than to rely on devs getting good. This is the code if conditional hooks were feasible-much more intuitive and readable:
// Much more self evident how to write the correct code.
// FWIW, it's still really opaque that we have a function that's secretly only running itself once based
// off hidden persisted state, but that's the price of admission of hooks. Once you drink the kool aid
// it's not so bad.
if (hasOptedIn == null) useEffect(() {
...
}, []);
While I think there are some reasonable ways to implement what is being asked here (see below), I think it is an example of XY problem.
The problem you are facing is that you want to express declarative properties (e.g. graph of dependencies) by intermixing these dependencies into imperative code. Then you encounter the problem that certain uses of these imperative APIs don't map into correct declarative properties - but you can't outlaw these uses because of how your APIs are designed. Now you are going to try to plaster it over using some other things, but in reality it means that maybe your API is actually wrong. APIs should be designed in a way that prevents incorrect uses.
I understand that some people find this sort of code appealing to write, but I think the code readability and maintainability would benefit if this code was actually properly partitioned, rather than intermixed and then magically separated using hidden properties - like order of execution of hooks.
What we could implement (but I am not sure we should)
Allow something like this on static functions, where SourceLocation is a final class defined in dart:core.
final class SourceLocation {
const SourceLocation._(...);
String? get fileUri;
int? line;
int? column;
// Special marker values
static const caller = SourceLocation._(...);
static const unknown = SourceLocation._(...);
}
void foo({SourceLocation location = .caller})
When compiling static calls: if named parameter of type SourceLocation is not passed explicitly and default value is SourceLocation.caller then inject caller: const SourceLocation._(/* some unique opaque description of location */) into the argument list.
There are some other variants where we could inject some canonical but mutable object instead - but I am not sure.
@mraleph sorry but I think that's not the correct classification of the problem. Hook like approaches are a valid way to make state management calls inside a Flutter build method easier and the code is much easier to read compared to using dozens of builders. Having a condititional in a build function too is super common. This would allow to remove that limitation.
I'm not sure how your proposal above would work, especially as you only reference static calls.