returns
returns copied to clipboard
dry-python
Think about typed do-notation
I have removed @pipeline in 0.14 release.
It had lots of problems:
- It was not working properly with
Failuretypes: #90 - It had problems with unwrapping complex values like
IOResultandFutureResult - It had problems unwrapping containers of different types (you were not allowed to unwrap both
MaybeandResultin the same@pipeline - It was encouraging imperative code in a functional codebase
I would love to have some solution instead! But, no ideas about how it should work or look like. So, any ideas are welcome.
Related: https://github.com/papaver/pyfnz/blob/master/pyfnz/either.py#L144
FWIW... Parsy (parser combinator library) has a way of mimicking do notation using generators and yield expressions:
https://parsy.readthedocs.io/en/latest/ref/generating.html#motivation-and-examples
their example:
@generate("form")
def form():
yield lparen
exprs = yield expr.many()
yield rparen
return exprs
if roughly equivalent to:
form = do
lparen
exprs <- expr.many()
rparen
exprs
...where form, lparen, rparen and expr.many() are all Parser monads
the correspondence works well enough, in this specific domain, that I was able to translate a bunch of code from Haskell's Megaparsec library into Python in a fairly straightforward way
usefully the generator/do form is not required, you can still use these objects in ordinary expressions like myparser = form | (lparen > other_token < rparen)
I don't know how similar or different this is to what you tried already, or if it's not useful, but if it's helpful at all then I'm glad 😄
@anentropic I have tried this approach. It works fine. But, it is impossible to type it properly.
Because, generators require to use Generator or Iterator return types. And these types require a specific generic parameter, ie: Generator[_SendType, _ReturnType, _YieldType]
When this parameter is specified, it becomes impossible to use other types inside your do-notation.
Currently I am thinking about this API:
assert do(
x + y
for x in wrap(Some(1))
for y in wrap(Success(2))
) == 3
But, this API still does not work properly with IO values. How can we unwrap IO in this comprehension safely?
It does mean that for z in wrap(IO(1)) should return raw 1 value, but the result of do(...) would have IO type on top of it.
Ideas?! 💯
@papaver since I feel very inspired by your implementation I would love to hear your opinion and ideas about it! 🙂
Related: https://github.com/papaver/pyfnz/blob/master/pyfnz/either.py
I am curious, I didn't fully understand the typing issue with generators+yield expressions.... and this looks like a generator expression as an argument to a function. Does it not have the same problem?
for x in wrap(Some(1)) seems kind of a cumbersome/weird way to extract a single value, compared to x = yield Some(1) (or maybe it needs to be wrapped... x = yield wrap(Some(1)))
(none of this is intended as a criticism, I am just curious about the issues involved)
@anentropic maybe I got your idea wrong. Can you please send a simple working demo?
@sobolevn i ended up going the generator route as it resulted in the cleanest code while still looking like python. all the other implementations i ran into were fairly gross looking and convoluted to force the code to do something it wasn't meant for (like the decorator with dozens of yields). the for comprehension was was able to do a form of 'lifting' for free, cleanly.
i actually use that pattern in code all the time now. most people just don't understand the power of the for comprehension.
# run validator on cut data, convert errors into issues as well
def tryValidation(validator, cutData):
idx, cut, take, shot = cutData
def toIssue(e):
return {
'issue' : 'ValidationIssue',
'cut_idx' : idx,
'shot' : shot and shot.code or None,
'take' : take and take.code or None,
'validator' : validator.__name__,
'error' : repr(e) }
return Try(validator, cutData).recover(toIssue)
# keep track of all the issues found
issues = []
def validateCutlist(cutlist, validators):
[issues.append(issue)
for validator in validators
for cutData in cutlist
for issue in tryValidation(validator, cutData)
if is_some(issue)]
personally i think typed python is a step backwards so i haven't messed with it. python is extremely powerful functionally without types.
We can also try to use something similar to Elixir's notation with with:
Like so:
with (
result as success_value,
needs_raw(success_value) as returns_raw,
):
print(returns_raw)
This is maybe obvious and the reason you chose it, but I like with and as because its consistent with how with is used here:
with open(file, "w+") as f:
...
which is also consistent with how Resource is used in scala cats and other monadic behavior. I'm not crazy about the needs_raw notation though and it would make sense to stick with bind to indicate flatmap like:
with (
fetch_api_keys() as apikeys
bind(get_user(apikeys)) as user
bind(get_settings(apikeys)) as settings
get_local_config() as local_config
do_stuff(user, settings, local_config)
) as final
Nope, this issue is not in the works right now. I like the generator expression approach:
do(
x + y + z
for x in c(Some(1))
for y in c(Some(2))
for z in c(Some(3))
)
With the support of async generators for Future.
Do you want to work on it?
Currently I am thinking about this API:
assert do( x + y for x in wrap(Some(1)) for y in wrap(Success(2)) ) == 3But, this API still does not work properly with
IOvalues. How can weunwrapIO in this comprehension safely? It does mean thatfor z in wrap(IO(1))should return raw1value, but the result ofdo(...)would haveIOtype on top of it.Ideas?! 💯
A couple of thoughts here:
- I don't believe you can typically mix different types of monads in the same do block (in pure functional languages that enforce strong static types). This is the problem that monad transformers solve. In this particular case, you'd probably just convert your Maybe value into a Result value and use 2 Result values in the do block.
- I think the result of
do(...)should be wrapped in IO. 3 is a pure value, butx + yhappens in the impure IO context. Once you enter IO, you cannot get back; you just have to keep working in IO. - what is
wrap? Is it just a way to turn a monadic type into a generator? I wonder if there's a way to bake generator functionality into the monadic types themselves so you could avoid thewrapfunction... - I think using generator expressions for do notation is a really cool idea. I don't know if they're equally powerful, but this reminds me of LINQ in C#. Language-Ext is a comprehensive C# library that uses LINQ for do notation, which could be a handy reference for this work.
Illustration in Haskell:
( do
x <- note "maybeX was Nothing!" (Just 1)
y <- Right 2
return (x + y) ) == Right 3
https://replit.com/@bagrounds/do-notation#main.hs
