returns
returns copied to clipboard
Request: Truthiness of Result, Correct way to filter over Iterable[Result]
Not sure if this is a question, feature request, recommendation, documenting request, or whatnot.
Love what you folks are doing with this library. However, I've been going through the docs and examples and I'm not sure how to handle Result/Option at the boundary with less-functional-style Python. Take the example of filtering over some failure-prone operation.
from returns.result import Result, Success, Failure
def half(x: int) -> Result:
if x & 1:
return Failure('not an even number')
return Success(x // 2)
results = [half(i) for i in range(5)]
results
out:
[<Success: 0>,
<Failure: not an even number>,
<Success: 1>,
<Failure: not an even number>,
<Success: 2>]
Alright, now I want to obtain only the successful results. My very first instinct is to reach for something like [el for el in results if el]
/ filter(lambda x: x, results)
, however Failure and Nothing are both Truthy. Alright, I can see how that interface makes sense, but then I want to reach for [el for el in results if el.ok]
, alas that doesn't exist.
Looking around the repo for inspiration, I find pipeline.is_successful
. Alright, that basically casts an UnwrapFailedError to a bool. I suppose I could do that, but seems rather clunky and not that performant. Alternatively, I could do bool(result.unwrap_or(False))
, but...ew. Checking isinstance(result, Failure)
fails cause Failure
is a function, and _Failure
is private and thus discouraged.
Ok there's Fold
surely there's from returns.iterables import Filter
....mmm nope. A list-like container with a .filter
method would also suffice, but I'm not seeing anything built-in. Maybe I'm supposed to use something like Fold.collect(results, ??)
? But I can't get the accumulator right. Fold.collect(results, Some(()))
just gives me a single Failure. I suspect I'd need some sort of iterable.List
which acts like list()
but supports the Container
interface.
Pipelining is all well and good, but in some places I want to start getting my feet wet with Maybes and Results, and it's a very shallow interface between lots of traditional imperative Python.
First, a question: What is the current canonical/recommended way to filter over a sequence of Results/Maybes? (see edit below)
My recommendations:
- I think the most pythonic and intuitive approach would be to define
__bool__()
for Nothing and Failure to return False. Nothing is very obviously Falsy, but Failure is a little less clear-cut, though my gut is it's Falsy. This gives the nice advantage offilter
works identically over Result and Maybe. Law of least surprisal. Etc. - If you aren't keen on that, definitely add some
.ok
property (preferred) or.ok()
method. I can't think of any reason not to. - It would also be cool to have
iterables.Filter
anditerables.ImmutableList
or eveniterables.List
if you want to be a filthy heathen, just to be able to.map()
over lists. -
Pair
container is actually pretty cool and I can think of a few uses. It would be nice if that were initerables
rather than just in the test suite.
I'd be willing to take a stab at writing up and PR-ing any of the above if you are interested.
Edit: Ok I'm a bit of a goofus, apparently I wanted Fold.collect_all(results, Success(()))
. This still seems a little roundabout. I still think a convenience method Fold.filter(results)
makes sense here, whether it's just sugar for Fold.collect_all(foo: List[Bar[Spam]], Bar(()))
, or checking for truthiness, or whatever.
Might also be good to have a bolded header for Filtering an iterable of containers just below Collecting an iterable of containers into a single container for, erm, the more oblivious developers out there ;)
Hi! Thanks a lot for the detailed issue! 👋
I agree, that is_successful
is not really the first choice.
I had several users confused about it, because it is kinda unrelated on the first sight.
I also agree that python
devs might expect bool()
to work on things like Result
. But, there are some hidden things here:
- It is not clear what we check with
bool()
, the container itself? Or its content? In other words:Success(False)
isTrue
orFalse
? - Performance wise adding a new method will be a good thing, but I really don't like to add a lot of custom methods to some containers. Why? Because it is hard to learn. Some containers might have (let's say)
.ok()
, some don't, it might not be clear why
About other features:
- I like your
Fold.filter
suggestion, but I guess it should work similar tofilter
and also accept optionalcallback
argument -
Pair
container is used somewhere as an example I think, you can surely add it to the core if you wish -
iterables.ImmutableList
and friends are also planned, see #257 Contributions are welcome!
iterables.ImmutableList and friends are also planned, see #257 Contributions are welcome! Exciting!
Some containers might have (let's say) .ok(), some don't, it might not be clear why
Definitely a good point.
It is not clear what we check with bool(), the container itself?
My intuition and least-surprisal is that Nothing
and Failure
are Falsy, everything else is Truthy, regardless of container contents. bool([False])
is true, after all. IO
and friends would always be True, so no change there. Failure
is Falsy because you can always call swap
and check the .failure()
method. And also by way of analogy to Some
/Nothing
. Both are effectively Either
s with a "happy path / sad path" dichotomy. bool()
here basically gets to whether the result is "happy" or "sad" so to speak.
But I also totally understand the sentiment of not wanting to make it too easy to interleave imperative-ish code with returns
style functional code, a la IOResultE
.
OH! I just discovered that unsafe.unsafe_perform_io()
works on Maybes and Results! That's kinda neat. I have taken to using Failure(SomeExceptionClass("error message"))
so this might give me the imperative escape-hatch I'm looking for.
I like your Fold.filter suggestion, but I guess it should work similar to filter and also accept optional callback argument
Do you mean like Fold.loop
's function
arg? I was thinking
def filter(predicate: Optional[Callable[[Any], bool], Iterable[hkt...] -> Iterable[hkt]: ...
where predicate
could be a regular unary function or an Applicative
(? still learning my terminology). A wrapped function.
OH! I just discovered that unsafe.unsafe_perform_io() works on Maybes and Results!
😮
where predicate could be a regular unary function
Yes, regular function.
😮
Lol, is that shocked face because you didn't know that function could do that? If so, that's some really good interface programming right there! To have that kind of symmetry over all containers is exactly why I am not sure adding __bool__
to some containers is the best approach. But also it unwraps to get the inner value, it doesn't tell you if the container is Truthy or Falsy.
OTOH, it seems like the way collect_all
filters Truth-ish containers is by how apply
behaves for Some/Nothing
and Result/Failure
, but for IOResult
, it's checking against
def apply(...)...
if isinstance(self, self.failure_type):
return self
if isinstance(container, self.success_type):
return self.from_result(
self._inner_value.map(
container.unwrap()._inner_value, # noqa: WPS437
),
)
So I would argue there is some precedent to add a __bool__ -> False
to a given Container
interface, since the container implementer would have to have some idea of the truthiness of their container, based on how they choose to implement apply
. I guess that moves the question to, "well if a container has some idea of truthiness based on whether apply
is implemented as a map or a no-op, shouldn't it just implement __bool__
as well?"
Regardless of any of that, I think a filter function would serve >80% of the needed use-case (simple and easy way to filter out Falsy values) without having to change the protocol of the container interface. Albeit, this is less efficient since it's having to call apply
, unwrap, map, concat, then rewrap, when it could simply call a vanilla filter
(see below).
So the default filter()
can take a None
as the predicate, in which case, the predicate is bool(obj)
, I believe. The docs don't actually specify this, but it's clearly evaluating the truthiness, which is:
By default, an object is considered true unless its class defines either a
__bool__()
method that returnsFalse
or a__len__()
method that returns zero, when called with the object.
So I would expect the predicate to be bool()
by default, which kinda points back to whether the container implements __bool__
or __len__
, which currently aren't implemented by any containers, but I suspect you might want at least __len__
for a NotEmptyList
....boy I've cracked open a bit of a Pandora's box, haven't I?
Well, I decided to profile it using pysnooper
, and it's definitely doing a lot of "unnecessary" operations just to figure out if it can apply
or not. So I think there's definitely some room for optimization there. Maybe this is an "optimization vs elegance" issue.
things = [Success('success1'),Success('success2'), Failure(Exception('fail1')), Failure(Exception('fail2'))]
acc = Success(())
@pysnooper.snoop(depth=6)
def doit():
return Fold.collect_all(things, acc)
doit()
I think the real question is, do constructions like filter(None, list_of_Result_containers)
/[el for el in list_of_Result_containers if el]
a) support the broad goal of making Python more Functional, b) without compromising the simple elegance of the existing library? I think (a) is an emphatic yes - these are the batteries-included Functional Pythonic idioms, and should be supported. b) is trickier, since now if you want consistency between bool()
and apply()
, you gotta make sure to implement __bool__
and/or __len__
.
But I think this will be a bridge that'll be have to be crossed if you implement NotEmptyList
and someone wants to call len()
on it ;)
I will have some time to think about all the points you've made. And then return with some feedback. P.S. It was a pleasure to read! 👍
it's way too late to change the existing API, but my two cents is that it would have been ideal to have Failure (and Nothing) define __bool__
as false, and success (and Some) define it as True. Intuitively this just seems "correct", and makes for a much nicer on-ramp into using monads in otherwise imperative code.