citus icon indicating copy to clipboard operation
citus copied to clipboard

Parallelize all arbitrary config setup scripts

Open JelteF opened this issue 2 years ago • 1 comments

When running only a single test using arbitrary-configs-base a large part of the time is spent in setup of the tables. These separate setup scripts can all be safely parallelized. On my machine this reduces the runtime of the following command from ~4.5 to ~3.5 seconds:

make -C src/test/regress/ check-arbitrary-base CONFIGS=PostgresConfig EXTRA_TESTS=prepared_statements_1

And the runtime of the runtime of the following command is reduced from ~24 seconds to ~14 seconds:

make -C src/test/regress/ check-arbitrary-base CONFIGS=CitusDefaultClusterConfig EXTRA_TESTS=prepared_statements_1

Initial attempt at fixing deadlock detection

The deadlock detector had a bug where it would detect deadlocks even when there were none. This could happen when the lock graph contains a combination of transactions had a distributed transaction id assigned, but also some transactions that did not.

Multiple different transactions without a transaction id were considered the same transaction (with transaction id 0). Thus a cycle would be instantly detected if two were waiting on eachother. However, due to the nature of the deadlock detector these transaction without distributed transaction ids would normally not be included in the lock graph at all. Only when a transaction that has a distributed transaction id was waiting on one without, would it be included.

JelteF avatar Feb 23 '22 12:02 JelteF

We merged #5842 and it might make sense to drop CI configs for PG12 after a rebase or merge

hanefi avatar Mar 24 '22 00:03 hanefi

Although this has some useful code in it, I don't expect it to merge soon. @JelteF do you plan to continue this? If not, let's close the PR?

onderkalaci avatar Oct 21 '22 15:10 onderkalaci