fastapi-crudrouter icon indicating copy to clipboard operation
fastapi-crudrouter copied to clipboard

Add schemathesis tests

Open DanLipsitt opened this issue 2 years ago • 7 comments

This PR adds automated Schemathesis tests to the test suite. Most of the tests currently fail, and I believe this is due to a bug in Schemathesis, described in https://github.com/schemathesis/schemathesis/issues/1335. I see that the author of Schemathesis has kindly offered to help get it working with fastapi-crudrouter. I'll update the schemathesis ticket and hopefully this PR will help both projects.

DanLipsitt avatar Feb 10 '22 01:02 DanLipsitt

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/flortz/fastapi-crudrouter/4RVwLWcGLBPGZCGrrRHMXEyThDAU
✅ Preview: https://fastapi-crudrouter-git-fork-danlipsitt-105schemat-b57fbd-flortz.vercel.app

vercel[bot] avatar Feb 10 '22 01:02 vercel[bot]

Current CI failure seems to be due to a bug in pytest 7.0.0 (https://github.com/pytest-dev/pytest/pull/9609). Run tests again when 7.0.1 is out (should be very soon) or work around it in dev.requirements.txt.

DanLipsitt avatar Feb 10 '22 16:02 DanLipsitt

I added the pytest workaround. Any remaining failures should be due to the Schemathesis bug mentioned above.

DanLipsitt avatar Feb 10 '22 16:02 DanLipsitt

Thanks for this @DanLipsitt, great work! 🚀

awtkns avatar Feb 18 '22 06:02 awtkns

Thank you so much @DanLipsitt for bringing this PR and reporting the issue to Schemathesis! @awtkns thank you for reviewing & approving the changes :)

I added a small suggestion in the comment above which makes the tests work. They took quite some time to run on my machine (~7 minutes) but showed quite a lot of things that might require attention. I'd also suggest adding data_generation_methods=schemathesis.DataGenerationMethod.all() to schemathsis.from_dict so it also executes negative test scenarios, however they are even slower.

Let me know if I can help with anything :) I'd be happy elaborate

Stranger6667 avatar Feb 21 '22 19:02 Stranger6667

Updated per @Stranger6667's suggestion.

DanLipsitt avatar Mar 23 '22 01:03 DanLipsitt

Someone is attempting to deploy a commit to a Personal Account owned by @awtkns on Vercel.

@awtkns first needs to authorize it.

vercel[bot] avatar Jan 29 '23 06:01 vercel[bot]