piccolo
piccolo copied to clipboard
piccolo-orm
Best practices for writing unit tests
In django each unit test is wrapped in a database transaction. It means that any changes made during test (e.g. calling an endpoint which adds a new object) are rolled back after the test ends.
This is very useful because the state of the DB does not change between tests, and assertions are easy to write and very easy to understand (like: I call an endpoint to create an object, then I check that there is exactly 1 object in the database).
Is there a way of achieving the same transactionality in piccolo and FastAPI ? Are there any examples?
You can create transactions using Piccolo.
# Access the engine somehow, easiest is via any Table class.
async with MyTable._meta.db.transaction():
# Everything here is now inside a transaction
Piccolo's transactions are async only. Are you using Pytest for fixtures? That might make things trickier, as you would need to wrap those in the same transaction.
You could try something like this:
class TestFinished(Exception):
pass
def my_test():
async def test():
try:
async with MyTable._meta.db.transaction():
setup()
response = test_client.get('/')
assert response.status_code == 200
raise TestFinished() # Raising an exception will stop the transaction from committing the changes
except TestFinished:
pass
asyncio.run(test())
The way I usually test my FastAPI apps is using fixtures like this:
@pytest.fixture
def user():
user = User.objects().create(username="bob").run_sync()
yield
user.remove().run_sync()
Or using IsolatedAsyncioTestCase from unittest.
Thank you for your answer and examples. Yes, I'm using pytest.
I'm aware Piccolo has transactions, but the documentation on transactions is very basic. My intention was to write a pytest fixture (function scope) which would wrap every test function in transaction, and then automatically roll it back at the end. Something similar to what django does.
Having such a wrapper would be super cool. Everything created during a test (either in other fixtures or in the test function) would nicely roll-back at the end of each test. The database would start clean before each test and end up clean after each test.
But I don't know how to do it, there just doesn't seem to be any nice and clean way of doing it. Plus I need to use FastAPI's synchronous test client , because I'm using its websocket features.
It might be possible to create an async fixture using pytest-asyncio.
https://pytest-asyncio.readthedocs.io/en/latest/reference.html#decorators
I'm not sure about the internals of the FastAPI / Starlette TestClient. You might be able to use the synchronous client within an async test.
For anyone looking for this, this is what I currently do to wrap each test in a transaction using pytest-asyncio.
Add this to conftest.py:
@pytest.fixture(scope="function", autouse=True)
async def piccolo_transaction(event_loop):
"""
It should be as simple as::
async with DB.transaction():
yield
However, pytest-asyncio doesn't pass down contextvars, which is how Piccolo
handles transactions.
https://github.com/pytest-dev/pytest-asyncio/issues/127
For now, this is the work around.
"""
DB: PostgresEngine = engine_finder()
connection = await DB.get_new_connection()
transaction = DB.transaction()
transaction.connection = connection
transaction.transaction = transaction.connection.transaction()
await transaction.begin()
class TransactionProxy:
def get(self):
return transaction
DB.current_transaction = TransactionProxy()
yield
await transaction.rollback()
await connection.close()
It's a bit of hack - it's because pytest-asyncio doesn't pass contextvars down from fixtures.
Tracking this in a separate issue: https://github.com/piccolo-orm/piccolo/issues/780
@sarvesh-deserve I don't think so, just because the transactions in Piccolo have to be async.
@sarvesh-deserve Depending on how complex your database schema is, you could create and tear down the tables after each test.
Here's an example using the setUp and tearDown methods of TestCase:
https://piccolo-orm.readthedocs.io/en/latest/piccolo/testing/index.html#creating-the-test-schema
Or alternatively, delete all the information from your test tables.
Yes, but it is increasing our test time in exponential manner. This is what we are doing:
@pytest.fixture(autouse=True) def clean_db(): models = Finder().get_table_classes() for model in models: asyncio.run(delete_rows(model))
@sarvesh-deserve If you're using SQLite to run your tests, you could try deleting the database file.
You could try converting your tests to be async using this.
Dropping the tables and recreating them isn't ideal, but it's not too terrible - we do that with the Piccolo test suite, and it runs hundreds of tests in a minute or so.
Hello, great library, I would like to ask how can I create a test database in postgres case in unit tests if does not exists. Is there any utils that can help for this or I should use raw sql queries.
@anryangelov I think using raw SQL is your best option. Piccolo doesn't really have anything for creating databases - it assumes they already exist.