lazy-fixture is not compatible with pytest 8

[jelly]

Pytest 8 no longer works with lazy-fixture, see https://github.com/TvoroG/pytest-lazy-fixture/issues/65, other projects which used it have switched away from it like pyarrow and httpie.

In Arch Linux we want to keep running the b2-sdk tests during the building of the package and will likely patch lazy-fixture to keep it working but would prefer dropping it from our repositories.

[polyzen]

There's also https://github.com/dev-petrov/pytest-lazy-fixtures that could be used instead.

[agoncharov-reef]

Thank you for the issue!

We definitely could modify sources to resolve this, but imo this should be addressed in different way. As I see, current PKGBUILD uses system-wide unpinned pytest installation to run tests which now leads to these conflicts. Even if we fix it now, nobody guarantees it won't break in next rolling release of Arch linux, either because of pytest or any other library breaking chages. Fixing lazy fixtures is curing symptoms, not eliminating root cause.

B2 cannot guarantee execution and correctness of tests for any versions except those defined in contract, which is pyproject.toml's test section in this case: https://github.com/Backblaze/b2-sdk-python/blob/master/pyproject.toml#L175

I think we should simply reproduce the test environment proposed by B2. I see that aws-cli-v2 package already uses venv to perform tests, we could do same here. So I see two solutions:

1. create venv as in aws-cli-v2, install test environment there (something like pip install -e .[test]) and run tests in this venv ("$PWD/venv/bin/python" -m pytest ...)
2. b2 uses nox for testing which takes care of creating venv and installing dependencies, and Arch has this package in repos, so we could just add nox and pdm (instead of pytest) as dependency for testing, and call nox -s "unit-3.11" in PKGBUILD. This is better imo, since it will be as close to internal B2 tests as possible (for example, it performs tests on multiple versions of library).

Wdyt?

[jelly]

Thank you for the issue!

We definitely could modify sources to resolve this, but imo this should be addressed in different way. As I see, current PKGBUILD uses system-wide unpinned pytest installation to run tests which now leads to these conflicts. Even if we fix it now, nobody guarantees it won't break in next rolling release of Arch linux, either because of pytest or any other library breaking chages. Fixing lazy fixtures is curing symptoms, not eliminating root cause.

Of course and we can introduce pytest 7 to resolve the situation. I created this issue originally to bring attention to the issue of the 2 year not updated and now broken pytest plugin.

B2 cannot guarantee execution and correctness of tests for any versions except those defined in contract, which is pyproject.toml's test section in this case: https://github.com/Backblaze/b2-sdk-python/blob/master/pyproject.toml#L175

Of course that is understandable.

I think we should simply reproduce the test environment proposed by B2. I see that aws-cli-v2 package already uses venv to perform tests, we could do same here. So I see two solutions:

1. create venv as in `aws-cli-v2`, install test environment there (something like `pip install -e .[test]`) and run tests in this venv (`"$PWD/venv/bin/python" -m pytest ...`)

If that uses the system python packages that works, otherwise we aren't really testing what we ship.

2. b2 uses [`nox`](https://nox.thea.codes/en/stable/index.html) for testing which takes care of creating venv and installing dependencies, and Arch [has this package in repos](https://gitlab.archlinux.org/archlinux/packaging/packages/python-nox), so we could just add `nox` and `pdm` (instead of `pytest`) as dependency for testing, and call `nox -s "unit-3.11"` in `PKGBUILD`. This is better imo, since it will be as close to internal B2 tests as possible (for example, [it performs tests on multiple versions of library](https://github.com/Backblaze/b2-sdk-python/blob/master/noxfile.py#L118)).

Wdyt?

Sure, we can solve this in Arch Linux. Just wanted to bring the pytest plugin situation to your attention.

[agoncharov-reef]

Yeah, installing python -m venv --system-site-packages "$PWD/venv" seems like a proper solution: it gonna use global Arch packages for real work during tests but local venv packages for running tests.

[mjurbanski-reef]

While we don't plan to work on this while it works in our CI setup, we are open to PRs replacing this test dependency with one that works with the newest pytest.

Something that must be mention here:

• https://github.com/dev-petrov/pytest-lazy-fixtures seem to be Python>=3.8 online, while we have yet to drop Python 3.7 ; We are open to dropping it, but if this would be the PR that drops it, we cannot guarantee it will be merged in the next month
• when new dependency is introduced, we need to review its whole codebase for security reasons, and check authors track record. Using packages of well know authors makes it more likely that it will be accepted. This is also reason why something "old" i.e. "mature" is easier for us to trust.

[loqs]

While pytest-lazy-fixtures is compatible with pytest 8, I encountered a scope issue after the switch with pytest 8``

==================================== ERRORS ====================================
________ ERROR at setup of TestCache.test_save_bucket[in_memory_cache] _________
ScopeMismatch: You tried to access the function scoped fixture in_memory_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:23:  def in_memory_cache()
_ ERROR at setup of TestCache.test_get_bucket_id_or_none_from_bucket_name[in_memory_cache] _
ScopeMismatch: You tried to access the function scoped fixture in_memory_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:23:  def in_memory_cache()
_ ERROR at setup of TestCache.test_get_bucket_name_or_none_from_bucket_id[in_memory_cache] _
ScopeMismatch: You tried to access the function scoped fixture in_memory_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:23:  def in_memory_cache()
___ ERROR at setup of TestCache.test_list_bucket_names_ids[in_memory_cache] ____
ScopeMismatch: You tried to access the function scoped fixture in_memory_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:23:  def in_memory_cache()
___ ERROR at setup of TestCache.test_set_bucket_name_cache[in_memory_cache] ____
ScopeMismatch: You tried to access the function scoped fixture in_memory_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:23:  def in_memory_cache()
________ ERROR at setup of TestCache.test_save_bucket[auth_info_cache] _________
ScopeMismatch: You tried to access the function scoped fixture auth_info_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:28:  def auth_info_cache()
_ ERROR at setup of TestCache.test_get_bucket_id_or_none_from_bucket_name[auth_info_cache] _
ScopeMismatch: You tried to access the function scoped fixture auth_info_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:28:  def auth_info_cache()
_ ERROR at setup of TestCache.test_get_bucket_name_or_none_from_bucket_id[auth_info_cache] _
ScopeMismatch: You tried to access the function scoped fixture auth_info_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:28:  def auth_info_cache()
___ ERROR at setup of TestCache.test_list_bucket_names_ids[auth_info_cache] ____
ScopeMismatch: You tried to access the function scoped fixture auth_info_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:28:  def auth_info_cache()
___ ERROR at setup of TestCache.test_set_bucket_name_cache[auth_info_cache] ____
ScopeMismatch: You tried to access the function scoped fixture auth_info_cache with a class scoped request object, involved factories:
test/unit/test_cache.py:33:  def cache(request)
test/unit/test_cache.py:28:  def auth_info_cache()

Is changing the scope an option. Would removing the use of pytest-lazy-fixture without replacement be preferable to switching?

[agoncharov-reef]

I think removing instead of replacement is fine.

[selfisekai]

submitted #512

[Hawk777]

I think this can be closed with the release of 2.6.0 can’t it?