Adding integration tests would prevent to do manual tests at each and every PR merge.
Whithout going the full tox route right from the start, this looks like a nice starting point: https://github.com/keeppythonweird/catinabox