Add support for pytest-reportlog

[gnikonorov]

pytest-reportlog now replaces _pytest.resultlog, and _pytest.resultlog is no longer available in pytest v6.1.0

#129 removes _pytest.resultlogis, but we should still work on integrating with the replacement.

[sallner]

The question is here, whether we really need it now. Maybe it already works out with the new implementation. Having a look at the old threads, the problem seemed to be specific to the implementation in pytest itself, so maybe we solved the problem by time now. #28 / #29 .

[ssbarnea]

pytest-html own ci was hit by the same change pytest master. I would recommend enabling a devel job on CI that runs on schedule and tests main dependencies from master.

Here is an example https://github.com/pytest-dev/pytest-html/blob/master/tox.ini#L24-L30 -- the list of packages to install from master is up to you but at least pytest makes sense for any pytest plugin.

I need to mention the that #129 test is broken as it fail if pytest is installed from master. Let me demonstrate it:

$ pip freeze|grep pytest                                                                                           pytest @ git+https://github.com/pytest-dev/pytest.git@b4c28dcaa25d8c93bea6124a1e2210a3cd7b5235

$ python                                                                                                           
>>> import pytest
>>> pytest.__version__
'6.0.1.dev299+gb4c28dcaa'

As you can guess this makes the code believe this version supports _pytest.resultlog but it does not. Instead of looking for specific versions it may prove more reliable to discover features using getattr, dont you think?

[sallner]

Puh, I agree with you, that we should test for feature instead of version.

But what I do not understand, is how you get this odd pytest version if you build it from master. We should be at 6.1.1 for some time now. But I actually cannot find any version number in the repository(https://github.com/pytest-dev/pytest/blob/master/src/_pytest/init.py), so it seems to be only added for the release process nowadays. https://github.com/pytest-dev/pytest/issues/7891 might be also talking partly about this issue.

[ssbarnea]

Wrong versioning is another issue that is up to pytest to deal with. In fact is quite a common one because even the smart versioning tools like setuptools-scm, have no idea when a major version is introduced unless you first create a tag, they all follow a very simple logic: last tag on same branch plus an increment, like dev123. I did not check but I am ready to bet that 6.1.0 tag was never on master branch.

I often do create a 2.0a0 kind of version as soon I merge the first major/breaking change to a project. These count as pre-releases so even if they go to pypi there is no issue. The real benefit is that any new merge made after would produce a correct version.

Maybe @nicoddemus @The-Compiler look at this and realise how useful is to assure latest releases (tags) are always from default branch. Is ok to have newer tags for old stable releases on other branches but default project branch should be the one receiving the last tag.

[hugovk]

I did not check but I am ready to bet that 6.1.0 tag was never on master branch.

Yep:

This happens because the 6.1.0 tag is only in the 6.1.x branch and setuptools-scm can't find it in master...

https://github.com/pytest-dev/pytest/pull/7805#issuecomment-700600258

[nicoddemus]

I would be OK with adding a 6.3.0.dev tag on master right after the first 6.2.0 release is made from 6.2.x, if that would avoid confusion.

@bluetech thoughts?

[bluetech]

@nicoddemus Let's do it! We'll need some way to make it not trigger a PyPI release.

[nicoddemus]

hmmm indeed, I guess we can make use of the regex features to not match tags with end in .dev?

[ssbarnea]

Almost all tools using git make the assumption that latest versions of a software are on the default branch. This makes sense and is natural, so instead of introducing more complexity I would suggest pytest to adopt this standard use of git.

This means that supported stabke versions should be in named branches, like stable/1.x and master/main should always contain the next version. It is easy an works nice with pip, editable installs and so on.

I would argue that 6.3.0a0 would be the correct tag to add, the dev tags are reserved for auto-numbering done by setuptools-scm and similar tools, those that build versions relative to last tag + number of commit after. These versions are not guaranteed to be portable as my local commit number may be different than yours.

[nicoddemus]

This means that supported stabke versions should be in named branches, like stable/1.x and master/main should always contain the next version. It is easy an works nice with pip, editable installs and so on.

That's the setup we have today already.

[BeyondEvil]

6.2.0-has-been-released-ping @nicoddemus 😛

[nicoddemus]

Hey @BeyondEvil,

Sorry, is there any action points I should tackle? I thought having the .dev tag was enough (pytest-dev/pytest#7908)?

[BeyondEvil]

Sorry about that @nicoddemus, I think I misread your previous comment. I thought you had to wait until after the release of 6.2.0 to merge the fix. 😊 🙏

[nicoddemus]

Ahh I see, no worries. 👍

[sallner]

Is currently anyone willing to work on the compatibility with the new pytest-reportlog?