hikari icon indicating copy to clipboard operation
hikari copied to clipboard

Type tests

Open mplatypus opened this issue 1 year ago • 6 comments

Summary

Add typing to all tests, along with a pipeline that forces tests to stay typed.

This will make writing new tests easier, and cleaner.

mplatypus avatar Mar 16 '25 13:03 mplatypus

This pr is going to be insanely hard to review, so I'm mostly going to base it on tests passing + the pipeline passing.

Any new questionable additions i will comment on as I slowly review this

Yeah, its all good. I know this is going to be a massive PR. I will mark things that I think you need to see with a FIXME: with some reason as to why I think its important to view and/or if I have questions about it.

mplatypus avatar Mar 16 '25 13:03 mplatypus

Okay, after doing some more changes, I ran nox and a few errors are occurring, but the weirdest part is, they only occur sometimes.

FAILED tests/hikari/impl/test_rest.py::TestRESTClientImplAsync::test_edit_guild - TypeError: An asyncio.Future, a coroutine or an awaitable is required
FAILED tests/hikari/impl/test_buckets.py::TestRESTBucketManager::test_start - TypeError: object MagicMock can't be used in 'await' expression
FAILED tests/hikari/impl/test_event_manager_base.py::TestEventStream::test___anext___waits_for_next_event - TypeError: An asyncio.Future, a coroutine or an awaitable is required

The above three tests are the only ones I have seen fail. Sometimes its two, sometimes its all three, and other times it just happily passes.

mplatypus avatar Mar 17 '25 07:03 mplatypus

they only occur sometimes.

Test order is randomized every time nox is ran, it's to catch flaxy tests or tests that behave differently based on what order they are run.

At the top of the pytest output you should see a seed flag with a random number next to it's it you pass it to nox like so: nox -s -- <flag_here> then you will be able to reproduce it consistently.

Here I would advice you to look at what order the tests are excuted and the most probable reason is that whatever is running before the first test is modifying the state of whatever is running afterwards.

If you provide me the flag, I can have a look too :)

I can also give a shot at reproducing it later

davfsa avatar Mar 17 '25 08:03 davfsa

I didn't know it did this. That's pretty useful.

Below are some seeds that succeed/fail when ran.

Successful seeds: 4147739088, 3310411651, 3105782950 Failure seeds: 4274378829, 1168526332, 620612017

mplatypus avatar Mar 17 '25 08:03 mplatypus

I didn't know it did this. That's pretty useful.

Below are some seeds that succeed/fail when ran.

Successful seeds: 4147739088, 3310411651, 3105782950 Failure seeds: 4274378829, 1168526332, 620612017

Fixed :)

davfsa avatar Mar 17 '25 12:03 davfsa

~~Todo: Change all typing.MutableMapping and typing.Mapping to use dict where applicable~~

Done: 26618df (#2213)

davfsa avatar Apr 08 '25 17:04 davfsa