GitHub CI test case fails, while test case passes locally

[max-baum]

Hi!

Has anyone else encountered an instance where a GitHub workflow test case fails, but the test case doesn't fail on the lambda server. Specifically:

This is the output of running docker compose exec pg ./run_tests.sh on the lambda server

sql/01.sql pass
sql/02.sql pass
sql/03.sql pass
sql/04.sql pass
sql/05b.sql pass
sql/05c.sql pass
sql/05d.sql pass
sql/05e.sql pass
sql/05f.sql pass
sql/05.sql pass
sql/06.sql pass
sql/07.sql pass
sql/08.sql pass
sql/10.sql pass
sql/11.sql pass
sql/12.sql pass
sql/13.sql pass
sql/14.sql pass
sql/15.sql pass

and this is the output of running the CI test cases on GitHub. See the test case for 13.sql

sql/01.sql pass
sql/02.sql pass
sql/03.sql pass
sql/04.sql pass
sql/05b.sql pass
sql/05c.sql pass
sql/05d.sql pass
sql/05e.sql pass
sql/05f.sql pass
sql/05.sql pass
sql/06.sql pass
sql/07.sql pass
sql/08.sql pass
sql/10.sql pass
sql/11.sql pass
sql/12.sql pass
sql/13.sql fail
sql/14.sql pass
sql/15.sql pass
Error: Process completed with exit code 2.

Thanks!

[mikeizbicki]

This means that you have some nondeterminism somewhere in your SQL query. The only (I think) possible source of nondeterminism in these queries is the ordering if the ORDER BY clause is not used. Most likely, the ordering selected by postgres on github is different than the ordering selected by postgres on the lamda server, and fixing the ORDER BY will make the test case pass.

The github actions should output a full diff of the failing test cases for you to verify that the problem is related to ordering.