Question: recommended approach for `PurePosixPath`-like class that is not `os.PathLike`

[carlcsaposs-canonical]

Hello,

We are trying to implement an interface similar to pathlib.Path for a remote posix filesystem (more specifically, to access files on the main Kubernetes container from a sidecar container via pebble)

Our initial thought was to subclass pathlib.PurePosixPath and implement a subset of pathlib.Path's API (e.g. write_text) ourselves

However, we noticed that PurePath is os.PathLike but we wanted instances of our class to raise a TypeError/fail a type checker if used in functions like shutil.move (same rationale as https://github.com/python/cpython/issues/106037)

---

To create a class that has a PurePosixPath-like interface without being os.PathLike, here are the approaches we're looking at

Approach 1: Using pathlib-abc

import posixpath

import pathlib_abc


class Foo(pathlib_abc.PurePathBase):
    parser = posixpath

    def __init__(self, *paths):
        paths = (path._raw_path if isinstance(path, Foo) else path for path in paths)
        super().__init__(*paths)

Approach 2: Using pathlib._abc from Python 3.14.0a4+

import pathlib
import pathlib._abc


class Foo(pathlib._abc.JoinablePath):
    def __init__(self, *args):
        args = (arg._path if isinstance(arg, Foo) else arg for arg in args)
        self._path = pathlib.PurePosixPath(*args)

    def with_segments(self, *pathsegments):
        return type(self)(*pathsegments)

    def __str__(self):
        return str(self._path)

With both of these approaches, one of the use cases we're trying to support is

Foo("fizz") / Foo("buzz")

---

Is this how you intended these classes to be subclassed, or would you recommend a different approach?

We read through https://discuss.python.org/t/make-pathlib-extensible/3428 and followed along to the best of our limited understanding & experience

We greatly appreciate all your work to improve pathlib and make it more extensible—it's clear that this is a monumentous effort and we're very grateful for & impressed by the improvements you've already made

With all of the active development (and the difference between pathlib-abc and pathlib._abc in https://github.com/python/cpython main), we were wondering—specifically and only for the use case of a PurePosixPath-like interface that is not os.PathLike—if the API is relatively stable and whether it would be a good idea to expose that to end users

For our use case, we are trying to provide python 3.8+ compatibility. We are not concerned about breaking changes that affect how we define a PurePosixPath-like class, we are only concerned about breaking end users that use (and potentially subclass) that class

To be clear, we are not asking you to guarantee that there won't be breaking changes—only trying to understand:

• if you anticipate breaking changes
• whether, in our pursuit of a PurePosixPath-like class that is not os.PathLike, it would be better—at this time—to:
  • inherit from pathlib-abc
  • or if it would be better to reimplement PurePath's interface from scratch (e.g. by making a class that uses PurePath for the implementation [e.g. self._path: PurePath] but doesn't inherit from PurePath)

Thank you for all your contributions to pathlib!

[barneygale]

Hey, thanks for reaching out. I'll write up a more thorough reply tomorrow, but for now I have a suggestion:

class Foo(pathlib.PurePosixPath):
    __fspath__ = None

This will produce a TypeError when someone calls os.fspath(Foo()), and in Python 3.13+ the exception message treats it as being unset: https://github.com/python/cpython/pull/106082

If this works for you, you might also consider setting __bytes__ and as_uri() to None

[carlcsaposs-canonical]

thank you!

I tried that earlier, and while it raises a TypeError at runtime, it does not appear to get picked up by my type checker (PyCharm)

I just tried it with mypy and pyright, and they also do not seem to pick it up e.g.

import pathlib
import shutil

class Foo(pathlib.PurePosixPath):
    __fspath__ = None

shutil.rmtree(Foo())

the type checker does not show an issue at shutil.rmtree

Compared to

import shutil

class Bar:
    pass

shutil.rmtree(Bar())

which results in PyCharm

Expected type 'str | bytes | PathLike[str] | PathLike[bytes]', got 'Bar' instead

mypy

foo.py:6: error: No overload variant of "__call__" of "_RmtreeType" matches argument type "Bar"  [call-overload]
foo.py:6: note: Possible overload variants:
foo.py:6: note:     def __call__(self, path: str | bytes | PathLike[str] | PathLike[bytes], ignore_errors: bool, onerror: Callable[[Callable[..., Any], str, tuple[type[BaseException], BaseException, TracebackType]], object], *, onexc: None = ..., dir_fd: int | None = ...) -> None
foo.py:6: note:     def __call__(self, path: str | bytes | PathLike[str] | PathLike[bytes], ignore_errors: bool = ..., *, onerror: Callable[[Callable[..., Any], str, tuple[type[BaseException], BaseException, TracebackType]], object], onexc: None = ..., dir_fd: int | None = ...) -> None
foo.py:6: note:     def __call__(self, path: str | bytes | PathLike[str] | PathLike[bytes], ignore_errors: bool = ..., *, onexc: Callable[[Callable[..., Any], str, BaseException], object] | None = ..., dir_fd: int | None = ...) -> None
Found 1 error in 1 file (checked 1 source file)

pyright

/home/carlcsaposs/repos/path-demo/path_demo/foo.py
  /home/carlcsaposs/repos/path-demo/path_demo/foo.py:6:15 - error: Argument of type "Bar" cannot be assigned to parameter "path" of type "StrOrBytesPath" in function "__call__"
    Type "Bar" is not assignable to type "StrOrBytesPath"
      "Bar" is not assignable to "str"
      "Bar" is not assignable to "bytes"
      "Bar" is incompatible with protocol "PathLike[str]"
        "__fspath__" is not present
      "Bar" is incompatible with protocol "PathLike[bytes]"
        "__fspath__" is not present (reportArgumentType)
1 error, 0 warnings, 0 informations 

For us, it might be worth the tradeoff on type hints so that we don't need to re-implement PurePosixPath's interface & worry about compatibility across Python versions (we had a first go at re-implementing the interface for python 3.8-3.13 here: https://github.com/carlcsaposs-canonical/path-demo/blob/e50edccdb99e80dcbc3738fb399709c324734ad7/path_demo/_main.py)

Although it would be nice to have compatibility with the type checkers

thank you for the suggestion!

[barneygale]

FWIW I've just released pathlib-abc 0.4.0 which brings in recent changes from CPython.

A POSIX-y path could be implemented like this:

class MyPath(JoinablePath):
    parser = posixpath

    def __init__(self, *pathsegments):
        segments = []
        for path in pathsegments:
            if isinstance(path, MyPath):
                segments.extend(path._segments)
            elif isinstance(path, str):
                segments.append(path)
            else:
                raise TypeError("Expected str or MyPath")
        self._segments = segments

    def __str__(self):
        if not self._segments:
            return ''
        return self.parser.join(*self._segments)

    def with_segments(self, *pathsegments):
        return type(self)(*pathsegments)

Is that any use?

[carlcsaposs-canonical]

thank you for the update!

since we're targeting python 3.8+ compatibility (to support Ubuntu 20.04 LTS), I don't think we'll be able to use it for now—but will keep an eye on this for the future

we're also trying to provide a unified API for local filesystem operations (with pathlib.PosixPath) and remote filesystem operations—which I think would require us to backport pathlib.PosixPath from 3.14 so that the local filesystem operations have a compatible interface with WriteablePath on python 3.8—although I suppose we could only use JoinablePath and re-implement the WriteablePath interface for 3.8 compatibility (with pathlib.PosixPath)

in case it's useful, here's more context about what we're currently doing: https://pypi.org/project/charmlibs-pathops/ https://canonical-charmlibs.readthedocs-hosted.com/reference/pathops/

cc @james-garner-canonical

[barneygale]

Hey @carlcsaposs-canonical, it should be possible to re-introduce support for Python 3.8 if that would still be useful for you?

we're also trying to provide a unified API for local filesystem operations (with pathlib.PosixPath) and remote filesystem operations—which I think would require us to backport pathlib.PosixPath from 3.14 so that the local filesystem operations have a compatible interface with WriteablePath on python 3.8—although I suppose we could only use JoinablePath and re-implement the WriteablePath interface for 3.8 compatibility (with pathlib.PosixPath)

I'm planning to publish such a package shortly :)

[carlcsaposs-canonical]

hi Barney!

sorry for the delay I was hoping to give a substantive answer in the upcoming couple weeks, but priorities have changed and I'm not sure when I'll be able to get back to this

I think we'll probably end up targeting python 3.10+ or higher when we get back to this

I'm planning to publish such a package shortly :)

awesome, thank you! hope to check it out when we get a chance