aiouv icon indicating copy to clipboard operation
aiouv copied to clipboard

aiouv tests suite status

Open ludovic-gasc opened this issue 9 years ago • 10 comments

It's almost done, I've fixed all tests I can. @haypo has planned to change aiotest to avoid to access to self.loop._selector attribute.

If you remove _selector attribute failures, only test_soon_stop_soon doesn't pass, but the test is a little bit mystic for me. For me, the result produced by aiouv is more logical.

If I understand correctly, aiotest is enough to validate PEP 3156. Could I remove runtests.py or some tests from AsyncIO are necessary ? Could I enable Travis-CI ?

ludovic-gasc avatar Feb 11 '15 23:02 ludovic-gasc

IMO we should put more tests in aiotest first.

For call_soon, stop, call_soon: the test validates asynioc spec. Maybe I misunderstood the spec and it should be discussed on the mailing list.

Does runtests.py currently? Orat least a few tests?

vstinner avatar Feb 12 '15 06:02 vstinner

Could I remove runtests.py or some tests from AsyncIO are necessary ?

In the past rose used to pass the entire asyincio testsuite. It shouldn't be hard to achieve that again. Maybe this can be left as a more thorough test suite though, aiotest is a good start.

Could I enable Travis-CI ?

Sure! Let me know if you need me to flip any switch.

saghul avatar Feb 12 '15 08:02 saghul

For call_soon, stop, call_soon: the test validates asynioc spec. Maybe I misunderstood the spec and it should be discussed on the mailing list.

I haven't looked into it, but IIRC call_soon is specified quite loosely, so I guess it's possible 2 implementations differ a little there, while both being compliant.

saghul avatar Feb 12 '15 08:02 saghul

I haven't looked into it, but IIRC call_soon is specified quite loosely, so I guess it's possible 2 implementations differ a little there, while both being compliant.

I write an email to the Tulip mailing list to ask.

vstinner avatar Feb 12 '15 12:02 vstinner

About runtests.py: having to clone the asyncio repository is not convinient. You can drop runtests.py right now, but reminder to replace it with something else later ;-)

The most important point is to no have regression.

vstinner avatar Feb 12 '15 12:02 vstinner

+1 to drop runtests.py From my point of view, it should be interesting if we could mutualise testing effort in maximum with aiotest. It will be a good help for others event loop to respect AsyncIO standard. @haypo Do you see some use cases to add tests for aiouv but not for aiotest ?

Maybe, it could be necessary to move some tests from asyncio to aiotest, but if I understand correctly, aiotest is already used to test asyncio, isn't it ?

ludovic-gasc avatar Feb 12 '15 13:02 ludovic-gasc

Ok, lets drop it then.

saghul avatar Feb 12 '15 14:02 saghul

@haypo Do you see some use cases to add tests for aiouv but not for aiotest ?

You can take a look at asyncio, it has +800 tests. But aiotest must not check any private attribute or other implementation details.

Maybe, it could be necessary to move some tests from asyncio to aiotest

Don't do that. Since asyncio is now part of Python, and Python tests must be included in Python source code, I prefer to leave asyncio as they are.

if I understand correctly, aiotest is already used to test asyncio, isn't it ?

I added aiotest to trollius and tulip. But Python cannot use it in its test suite since Python don't use external dependencies.

Most asyncio tests are very specific to the implementation.

aiotest should be written differently. So both test suite are complementary.

vstinner avatar Feb 12 '15 14:02 vstinner

Ok, my comprehension: aiouv public API testing => aiotest aiouv private API testing => we need to have a test suite. Anybody has an objection to use py.test ? I already use that at work, we migrated from nose to py.test one year ago.

ludovic-gasc avatar Feb 12 '15 16:02 ludovic-gasc

2015-02-12 17:59 GMT+01:00 Ludovic Gasc [email protected]:

aiouv private API testing => we need to have a test suite. Anybody has an objection to use py.test ? I already use that at work, we migrated from nose to py.test one year ago.

I'm not sure that tests on the private API are need. But it's usually much easier to test the private API than testing the behaviour.

I like unittest.TestCase. nose, py.test and other tools (trial?) support unittest.

vstinner avatar Feb 12 '15 20:02 vstinner