trio
trio copied to clipboard
python-trio
Should we rename nurseries?
OK, let's do this.
Question: Should we rename "nurseries"? It's something that keeps coming up, and while it would be somewhat painful to change now, it'll be pretty much impossible within the next 6 months... so I guess now is the time to have a discussion, and if we decide not to change it then this can at least become the canonical issue that we link people to when they ask about it.
The most common version of this complaint is something like: "ugh, nursery is so cutesy. What about TaskGroup, or WaitGroup, or TaskScope, or something like that?" I don't find this version of the complaint compelling, so let me start by explaining why that is.
Names like TaskGroup, or TaskScope are excellent names for ordinary classes, the kinds of name we invent every day. For that kind of class, you want something that's descriptive, because you have a lot of them and people don't spend much time with any given one. And of course you want to follow the conventions, so the camelcase itself is informative: even if you aren't familiar with the particular class in question, you at least can tell at a glance that it is a class.
Nurseries, though, are a different kind of thing entirely. They're a new fundamental concept: you have to learn them up front, like you have to learn about for loops, and functions, and the difference between an object attribute and a local variable. Once you've used them for a few days, they recede into the background as part of your basic vocabulary. And they're just different from the concepts most people already know, so trying to give them a familiar descriptive name is actually misleading: you have to look them up and learn them. So, I want a name that feels like a primitive, like "for" or "function" or "variable" – something short and lowercase. And if anything, something that's a bit opaque and unfamiliar is probably better, because it signals "hey, new concept here". You can't have a lot of concepts like that, but nurseries are carrying an entire programming paradigm on their shoulders, so I think 1 new word is within our budget.
And regarding it being "cutesy": This doesn't really bother me, given that (1) this is a domain where "reaping orphan zombie children" is already considered totally ordinary technical language, and (2) after you use nurseries for a few minutes the name stops feeling cutesy, just like you stopped noticing that "trees" have nothing to do with botany, "functions" often don't function (little debugging joke there), "threads" have nothing to do with textiles, etc. So like... ok, it's a bit cutesy but whatever, people will get over it. And being cutesy actually has some upside: the whole joke about "a nursery is where child tasks live" probably does help it stick in people's minds. OTOH, I'm not like super attached to having a cutesy name either; I think it's mostly a neutral attribute.
So that's why I'm convinced that "nursery" is better than TaskGroup or similar.
But...... when talking to @graydon about this today, I did have some doubts about "nursery", for two reasons. The first is, he pointed out that "nursery" is also used as jargon for generational garbage collectors (to refer to the youngest generation, which often gets special handling). So a language implementer might ask "is this task in that nursery?", and "is this Task in the nursery?" and those are two completely unrelated questions. Fortunately it's mostly implementers who encounter the GC version of nursery, but still... this is a legitimate collision.
And the other is, I realized that the name is actually a bit less apropos since #375 landed and we tweaked the nursery semantics. Originally, I really wanted to emphasize the idea of "supervision", because we had the whole "parenting is a full time job" rule where the parent task had to park itself at the end of the nursery block or exceptions wouldn't propagate and stuff. Now the supervision part has dissolved into becoming an implicit part of the runtime, and doesn't really exist as a concept at all anymore – it's just, like, exceptions propagate themselves, what is there to talk about. The code inside the nursery block is just a slightly-special child. So that part of the metaphor has become a bit weird.
So here's another idea: maybe we could use a name that emphasizes that this is a place where your call stack splits/branches/words like that. Like, async with trio.open_split(), or open_branchpoint, or something like that? With the metaphor being that first you make a note "here's a point in my call stack where child tasks might split off", and then calling start_soon actually attaches a new branch there. I'm imagining like a hinge or a hook or something.
Anyway, that's just one idea – if y'all have other ideas I'd be interested to hear them, and ditto any thoughts on how the "nursery" name worked for you as either a newcomer to Trio or as you've become more experienced.
The nursery name works fine for me. It's well explained in the tutorial, and I liked the cuteness! 😄
split and branchpoint work too, but how would you call the resulting object? I think nursery.start_soon(fn) works slightly better than split.start_soon(fn)? Or is it because I'm used to nurseries?
Anyway, I'm not personally opposed to a change, as it would be easy to adapt to a new name.
What do you think of childminder?
To me it has a lot of the same helpful properties as nursery:
- similarly signals "important new concept - pay attention to this!"
- free-associates to watching over children equally well: https://en.wiktionary.org/wiki/childminder
...but isn't (to my knowledge) overloaded with another programming meaning, and also directly refers to the existing-programming-concept "child" in the name, and so maybe avoids some of the cutesiness that some people have found distracting?
That said, I'd be fine with keeping nursery and fine with a change too. </$.02!>
While I kind of like nursery, just for the heck of it, here's my random proposal:
If parenting isn't a full-time job anymore … maybe it's actually fun? Like a party?
So let's trio.start_party() and invite tasks to party together w/ party.invite(task) ?
Not sure about open_split. The word "split" is overloaded, and so doesn't necessarily invoke any obviously-intended associations immediately. Further overloading an overloaded term can add to cognitive load. It also doesn't signal "learn this important foreign concept!" to me as much.
The term "branchpoint" is better than "split" on both counts. But both "split" and "branchpoint" give up direct association with the concept of "children".
@apexo You still can't leave the party. You (as the "host", i.e. the nursery's initial task) can just decide to do your own thing while the party is ongoing, just like every other party guest.
My preferred name, should we decide to switch the name, would be the rather boring "task manager", or taskmgr.
That being said, I'm also OK with keeping the name. "Branchpoint" … well, maybe. @jab it does for me.
Personally: when I came across this nursery thing, I thought, OK, new name, new concept I actually need to think about instead of assuming that I already know what it's about. Which is exactly what we need to get across.
I didn't know about nurseries WRT garbage collection. However our nursery is very obviously not about GC, so I don't think that in practice there's much confusion. I'd be more wary about a name like branch since in if FOO then BAR else BAZ, BAR and BAZ are different branches of execution – a concept which (a) many people have already come across and (b) is in roughly the same realm, so more danger of confusion.
I like nursery
And the other is, I realized that the name is actually a bit less apropos since #375 landed and we tweaked the nursery semantics.
I think the name is still valid even after #375. Before this change we could see the main coroutine which open the nursery as the nurse which will take care of the child-coroutines. After the change the main coroutine is just a lazy parent who drops it children-coroutines to the nursery and go ~~have a beer~~ do it own work.
Attempting to make a positive contribution (browsing thesaurus looking for new names is a favourite pass-time anyways) I might suggest:
-
supervision-related:
- custodian
- steward
- caretaker
- porter
- attendant
- concierge
-
childcare-related:
- creche
- cradle
-
tree/splitting-related:
- outgrowth
- bifurcation
- partition
- branch
- sprout <-- personal fave
- offshoot
- bough
- limb
-
organization-division related:
- subsidiary
- office
- branch, also
- division
- department
- chapter <-- has a nice "part of a story" secondary resonance
My issue with options like sprout/bough/limb is that those sound like they refer to the children, while we need a name for the point where the children are attached.
@smurfix has a good point about "branch" already being overloaded, and as a control flow thing, so that's definitely confusing.
Of the nursery alternatives... maybe "splitpoint" is the best I can come up with so far? (As in, "this is a point where control can split".) It has some resonance with "checkpoint", which is our other bit of funny terminology. Something like:
async with trio.open_splitpoint() as splitpoint:
splitpoint.start_soon(...)
I actually kind of like how this de-emphasizes the concept of "children", since trio doesn't really reify tasks/threads/whatever-you-wanna-call-them. There are no task ids or task objects or anything (unless you go digging into trio.hazmat). OTOH we still need to refer to the different concurrent functions in conversation -- I suppose we could call them "splits", like: "While the first split is doing name resolution, the other splits are waiting...". Maybe that's too weird?
("sproutpoint" is also an option, but that might be too silly even for me :-).)
I suppose there's also
async with trio.open_family() as family:
family.start_soon(...)
but I feel like "family" is too commonly used as a kind of generic word for "category". "This family of concurrency frameworks uses families...", ick.
Seriously though, I thought nurseries was a little cutsie at first, but it didn't really bother me [1]_.
Now, I actually think it's a very good name and better than any alternative I've seen / thought of.
.. [1]: Full Disclosure: I use a data processing library called "pandas" on a daily basis and that also doesn't bother me! 😜
My problem with splitpoint or similar pointy names is that a point, conceptually, is just there. Like in geometry, it doesn't do anything.
My personal favorite still is a task manager. Maybe with trio.manage_subtasks() as mgr: await mgr.start(…).
My problem with
splitpointor similar pointy names is that a point, conceptually, is just there. Like in geometry, it doesn't do anything.
That would be the point though :-). (No pun intended.) When you type open_splitpoint, you'd be marking a place in the call stack where you could potentially hang off other children/sprouts/whatever:
f1 -> f2 -> (splitpoint) -> f3
This doesn't actually do anything. Then later, when you call start_soon, that's when something actually happens:
f1 -> f2 -> (splitpoint) -> f3
|
+--> f4
Well, in geometry you (mostly) don't just arbitrarily draw a point – the point exists as a consequence of something else (like two lines intersecting). Afterwards, other things exist as a consequence of the point being there (like a line being defined by two points). In both cases the "active" parts are the non-points.
shrug maybe I'm overthinking this … on reflection I'm like -0.1 on splitpoints-or-whatever, -1 on branchwhatever, and +0.1 on keeping nursery.
To me, splits/branches/sprouts suggest the point where things go different ways but not so much that they eventually merge. When thinking in spacial terms they might be the same thing, especially if you're used to visualize tasks in a tree, but it doesn't translate well to the idea of concurrency (multiple things running at the same time over parallel lines).
Maybe, given our peculiar semantics, it makes more sense to call the nursery a merge/join/muster point and use splitting/branching/forking/sprouting to refer to what now is the nursery.start operation.
The idea of supervision is lost but, well, so it is in the paradigm of calling/returning functions (either sync or async) which in some ways is a close relative to this. Instead of waiting for one function call to return or raise, you're waiting for many concurrent ones. What's so special about that? 😉
splits/branches/sprouts suggest the point where things go different ways but not so much that they eventually merge. ... it doesn't translate well to the idea of concurrency (multiple things running at the same time over parallel lines).
This is why I highlighted above the term chapter as having both organizational-structure gloss and a part-of-a-sequential-book gloss: the more I think about this the more I think it might be a good (mixed) metaphor. In the org-chart-sense it's definitely part-of-a-hierarchy. But in the book sense it's a block-structuring element of a sequential story that introduces-and-concludes a bunch of specific scenes, and maybe lets a couple threads of the plot connect-up with those from earlier or later chapters.
It also includes (unlike all the options here -- including nursery!) the concept of conclusion, as you say. The term evokes both "beginning of chapter" and "end of chapter" -- things started and things wrapped-up -- in almost equal measure.
I like my concurrency libraries like I like my concrete. Bland and functional. TaskManager
To use it in a sentence:
"The TaskManager manages its tasks."
Inessential weirdness and all that.
A common/boring name for a collection of jobs is a batch.
If you want a name that reflects some control-flow analogy...
Collections of threadlike things bound together at both ends are
- bundles
- fascicles (okay, that's weird, but maybe? They're hierarchical bundles of bundles of.., which is pretty cool)
- sheaves (maybe confusing if someone knows the mathematical definition of sheaf, which I don't)
Places that things leave and then return to
- nest (keep the cutesy!)
- camp
Portmanteau silliness: daglet
chapter really works for me. It's short, concise, and builds on a well-known literary concept with an emphasis on beginning and ending, with stuff happening in between. It doesn't assume anything about the nature of the story--whether child labor or worker robots--and it makes sense to talk about passing a chapter around to functions, e.g. "Which chapter [of the overall book/app] am I in?".
I have no issues with nursery, this is a python library and playfully named libraries are commonplace.
Anyway, nobody is shocked in this context about parents, children, (and infanticidal moves as killing children!), nursery is just a little blip that stops felling weird after you have seen it 12 times.
One analogy that has not been raised yet is the hub/spokes, but I am not fully convinced myself...
I like the name "nursery". If we choose a new one, I'd hope that it share some of the properties of "nursery" that make me like it:
- not in common usage elsewhere (as mentioned, maybe we can do even better than "nursery" in this regard, but I don't think the GC term overloading is a substantial issue)
- short and easy to type (I think seven characters is substantially better even than the ten in "splitpoint")
- doesn't invite ambiguous abbreviations (
mgrcould be a ThingManager for just about any value of Thing, and a large program might have a lot of those) - reads well in the trio pattern of
open_thing()being a function that returns a context manager that scopes the lifetime of a newthing - not too cutesy (I think "party" might be going a bit too far in that direction)
- suits the context in which it's used reasonably well - it doesn't have to be obvious without an explanation, but the explanation should make sense
I'm not sure it makes sense to switch for a name that's only infinitesimally better than "nursery", since there are switching costs. I don't feel like any of the names proposed so far are substantially better than "nursery", though I do like several of them.
Some random further thoughts:
- "custodian" is in common usage in Scheme, I think, and refers to a sort of scope for all resources - when a custodian is shut down, all files created in its scope are closed, threads are killed, etc. IMO it's similar enough to the nursery idea to make the differences a potential point of confusion for Schemers.
- The unit-of-an-organization meaning of "chapter" doesn't jump out at me nearly as much as the book-subdivision one does, so it feels like a mixed metaphor - the instinctive objection is "my program isn't made of chapters, that's for books! programs are made of (modules|packages|functions|...)". Nursery doesn't have this problem because nobody talks about dividing things into nurseries, in any domain.
- I quite like "hive" and "nest", and they're very short and non-overloaded (as long as no one tries to write a trio library for controlling a certain trendy home thermostat...), but I think they score worse on "likely preexisting associations link to correct concepts" than nursery does.
I don't really mind the nursery naming. If we do change it I wonder about a transportation analogy. Buses/trains/airlines tend to have hubs or central dispatches that buses/trains/planes leave from and eventually come back to. Something along those lines has the connotations of management, coordination, or authority.
Just an aesthetical subjective opinion from an amateur philologist, I would call them 'beacons' or 'cocoons' if you want to keep a biological-like sounding. If you want a more techie-industrial-crafting narration, maybe 'foundries' or 'workshops' it is more appropriate. A potentially helpful note, the whole library's concept and the arrows-like diagrams in your post made me think much about 3D-printing.
Thanks for your work.
The name nursery feels very dynamic and apt. It represents progress (growth) and chaos that is being constantly monitored on behalf of the parent process, which was responsible for the creation or the split. The metaphor is great - parent can just let the nursery manage the process while they take care of their work, with the how's of nursery being a black-box. Also, the name nursery stands out in the excellent blog post as something familiar w.r.t concurrency in everyday life, yet something I would want to read more about in the context of Trio. Irrespective of the simplifications in the implementations, the core idea behind a nursery resonates almost immediately for a first time reader and makes it approachable to take their first steps.
A coworker of mine suggested spool.
One could make the obvious connections with threads/threading and, if we stretch it, (sp)awn p(ool).
One problem with the name "nursery" is it only conveys that this is a place where new tasks can be spawned. But the really important thing about a trio nursery as I understand it, and in particular the thing that distinguishes it from other libraries' APIs for launching concurrent tasks, is that it also bounds the execution of all the tasks launched within it. That is, the nursery doesn't exit until all its child tasks have exited. The "nursery" metaphor doesn't convey this, in fact quite the opposite: most things born in nurseries grow up and leave the nursery to live their adult lives (especially if the nursery is working well and not prematurely killing it's charges :-) ).
I would favor a name that covey's this "boundedness", that no task started by this object will outlive that object. But I don't really have a good suggestion. I like variants of "scope", but @njsmith already rejected "TaskScope". I also like "lifetime", but that doesn't have the whimsy of the current name.
Here's a maybe off topic question though. Does there need to be a nursery object at all? Since the boundary provided by a nursery is the same as the async with statement (which is a lexical scope that doesn't need a name) couldn't trio just keep track of the current "nursery" (or whatever it's called internally) for the current task, and then let any async function start new tasks using global functions (e.g. trio.start and trio.start_soon)? It would seem to me the only change this would require compared to the current behavior is to have a top-level nursery created for the trio.run call itself, which would bound the execution of all tasks started within that top-level call. You would still be able to open new nurseries to create bounded groups of tasks within larger tasks, but you wouldn't need to name a local variable for the resulting object nor pass that variable to other functions, so the name would be much less important.
@maffoo - giving the nursery object a name makes it possible to give another task the ability to start tasks in your scope, even though it is not in that scope, by passing the other task the nursery object. This provides additional flexibility in more intricate cases, which I wouldn't want to do away with. (It may even be required by trio's internals, I'm not sure.)