aiopath
aiopath copied to clipboard
alexdelorenzo
Python 3.10 alpha 7 introduces new changes to pathlib
The following works on alpha 6 and below, but alpha 7 introduces this error:
In [1]: from aiopath import AsyncPath
In [2]: home = await AsyncPath.home()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-2-ef16332cd56b> in <module>
----> 1 home = await AsyncPath.home()
~/.pyenv/versions/3.10-dev/lib/python3.10/site-packages/aiopath/path.py in home(cls)
391 returned by os.path.expanduser('~')).
392 """
--> 393 coro = cls()._flavour.gethomedir(None)
394 homedir: str = await coro
395
~/.pyenv/versions/3.10-dev/lib/python3.10/site-packages/aiopath/path.py in __new__(cls, *args, **kwargs)
133 cls = AsyncWindowsPath if os.name == 'nt' else AsyncPosixPath
134
--> 135 self = cls._from_parts(args, init=False)
136
137 if not self._flavour.is_supported:
TypeError: PurePath._from_parts() got an unexpected keyword argument 'init'
The specific error in this ticket is fixed in this branch: https://github.com/alexdelorenzo/aiopath/tree/Python-3.10
Versions 0.6.x on PyPI work with Python 3.10a7: python3 -m install aiopath==0.6.1
Using the latest branch of pytest (via https://github.com/pytest-dev/pytest) with 3.10 support, it appears that the coroutine method AsyncPath.resolve() is broken.
______________________________________________________________________________ test_home ______________________________________________________________________________
@pytest.mark.asyncio
async def test_home():
home = Path.home()
ahome = await AsyncPath.home()
> await _test_is(home, ahome)
tests.py:91:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests.py:67: in _test_is
await _test_is(path.parent, apath.parent, test_parent=False)
tests.py:74: in _test_is
await apath.resolve(),
aiopath/path.py:491: in resolve
s: Optional[str] = await self._flavour.resolve(self, strict=strict)
aiopath/flavours.py:90: in resolve
result = await _resolve(base, str(path))
aiopath/flavours.py:73: in _resolve
target = await accessor.readlink(newpath)
../../../.pyenv/versions/3.10-dev/lib/python3.10/site-packages/aiofiles/os.py:13: in run
return await loop.run_in_executor(executor, pfunc)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = None
def run(self):
if not self.future.set_running_or_notify_cancel():
return
try:
> result = self.fn(*self.args, **self.kwargs)
E TypeError: readlink() takes exactly 1 positional argument (2 given)
../../../.pyenv/versions/3.10-dev/lib/python3.10/concurrent/futures/thread.py:52: TypeError
Here is a list of changes to the upstream pathlib package from the 3.10 alpha 7 stdlib:
- https://github.com/python/cpython/pull/19220
- https://github.com/python/cpython/pull/19481
- https://github.com/python/cpython/pull/18838
- https://github.com/python/cpython/pull/19342
- https://github.com/python/cpython/pull/18864
- https://github.com/python/cpython/pull/24294
- https://github.com/python/cpython/pull/18834
- https://github.com/python/cpython/pull/18841
- https://github.com/python/cpython/pull/25240
Full list of changes here: https://github.com/python/cpython/commits/master/Lib/pathlib.py
These changes will need to be ported to aiopath's Python-3.10 branch.
Just some notes for myself:
I started this project as a fork of pathlib for an opportunity to port synchronous code to asynchronous Python and to work with some standard library code. I wanted to keep the same class hierarchy and API to address use cases that might consume those interfaces, even if I personally don't plan on using them.
It might be worth profiling trio's approach of just wrapping synchronous I/O pathlib methods with decorators that submit the methods to a threadpool in the background: https://github.com/python-trio/trio/blob/6754c74eacfad9cc5c92d5c24727a2f3b620624e/trio/_path.py
That way I'm not beholden to upstream maintainers' changes in the stdlib, and as long as the API stays the same, aiopath won't have to change much if at all between changes to the stdlib.
The downside is punting some CPU-bound code to the threadpool, which, all in all, probably has an insignificant overhead.
It can also introduce problems like these:
- https://github.com/python-trio/trio/issues?q=is%3Aissue+pathlib+is%3Aopen
- https://github.com/python-trio/trio/issues?q=is%3Aissue+pathlib+is%3Aclosed
However, Barney Gale's updates to pathlib that appeared in 3.10 alpha 7 are stepping stones on a roadmap towards an extensible pathlib: https://discuss.python.org/t/make-pathlib-extensible/3428/
In a world with an extensible pathlib, it makes sense for aiopath to have the current class hierarchy it has (with AsyncPath subclassing AsyncPurePath etc). I'd also assume in such a world, pathlib would also have a stable interface to support consumers like aiopath.
I'm sorry my changes have broken things! As always with extending private APIs in the standard library, it can be a bit of a minefield. As you mentioned, I'm working towards a pathlib.AbstractPath class which supports subclassing. I'm hoping this can land in Python 3.11, all being well :)
