What's missing for feature parity with standard Odoo testing machinery?

[yajo]

This is more a question than an issue, but it could become a feature request if feature parity is feasible.

AFAICS pytest-odoo had a gap to bridge years ago: running tests without installing or updating addons: https://github.com/camptocamp/pytest-odoo/blob/9315331c4f0f33379bec4383fc272e57eeec16a2/README.rst?plain=1#L7-L13

Those sentences are no longer true. You can achieve the same with pure Odoo:

• Run it with --test-file ./addon/tests/test_this.py and it won't install or update, only test.
• Run it with --test-tags .test_name and only the test with that name will run.

So, we could say that the main goal of pytest-odoo is no longer an issue.

However, it still has the benefits of having the whole pytest ecosystem:

• It can generate coverage reports easily.
• It produces nicer output. Extremely nicer.
• It can produce XML reports.
• Nice IDE integration for test detection, execution and debugging.
• Probably more plugins that I'm forgetting right now.

I pushed https://github.com/odoo/odoo/pull/151728 some time ago to address these issues directly in Odoo, but it's clearly doomed for oblivion. So I'd be more than happy to entirely replace odoo testing machinery with pytest-odoo. But then...

https://github.com/camptocamp/pytest-odoo/blob/9315331c4f0f33379bec4383fc272e57eeec16a2/README.rst?plain=1#L17

Sadness.

People out there seem to ignore this warning. I'd like to ignore it too if possible. But I love my CI too much to feed it with unhealthy food.

So... what's missing to be able to remove that warning from the README? What's exactly the difference between pytest-odoo and odoo --test-enable that makes it not suitable for CI workloads?

IIUC these 2 points:

1. Lack of subtest support in pytest. This was true when pytest-odoo was born, but nowadays pytest-subtests should cover this. Does it work with Odoo? Has anybody tested it?
2. Unability to install or update addons, testing them under their canonical conditions.

Is it possible to reach feature parity and then reliably use pytest-odoo on CI workloads, entirely replacing (or wrapping) odoo?

Thanks!

@moduon MT-1075