CI - Clear `.spacetime` before running benchmarks

[bfops]

Description of Changes

If/when we have ABI breaking changes, this directory can cause benchmarks to start failing.

API and ABI breaking changes

None

Expected complexity level and risk

1

Testing

Describe any testing you've done, and any testing you'd like your reviewers to do, so that you're confident that all the changes work as expected!

• [x] Write a test you've completed here.
• [ ] Write a test you want a reviewer to do here, so they can check it off when they're satisfied.

[bfops]

benchmarks please

[github-actions[bot]]

Benchmarking failed. Please check the workflow run for details.

[github-actions[bot]]

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind, an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	5138	5138	0.00%	5176	5176	0.00%
sqlite	5528	5528	0.00%	6026	6026	0.00%

callgrind: filter

db	schema	indices	count	preload	_column	data_type	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	no_index	64	128	1	u64	77841	77841	0.00%	78297	78297	0.00%
stdb_raw	u32_u64_str	no_index	64	128	2	string	120511	120511	0.00%	121127	121127	0.00%
stdb_raw	u32_u64_str	btree_each_column	64	128	1	u64	23072	23072	0.00%	23480	23480	0.00%
stdb_raw	u32_u64_str	btree_each_column	64	128	2	string	24113	24111	0.01%	24597	24599	-0.01%
sqlite	u32_u64_str	no_index	64	128	2	string	143491	143491	0.00%	145113	145109	0.00%
sqlite	u32_u64_str	no_index	64	128	1	u64	122842	122842	0.00%	124242	124242	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	2	string	133290	133305	-0.01%	134940	134955	-0.01%
sqlite	u32_u64_str	btree_each_column	64	128	1	u64	130149	130149	0.00%	131639	131639	0.00%

callgrind: insert bulk

db	schema	indices	count	preload	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	64	128	867413	866961	0.05%	887309	886841	0.05%
stdb_raw	u32_u64_str	btree_each_column	64	128	1008463	1008533	-0.01%	1040483	1040641	-0.02%
sqlite	u32_u64_str	unique_0	64	128	396139	396139	0.00%	414227	414227	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	981456	981471	-0.00%	1023036	1023059	-0.00%

callgrind: iterate

db	schema	indices	count	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	1024	151301	151301	0.00%	151343	151343	0.00%
stdb_raw	u32_u64_str	unique_0	64	15286	15286	0.00%	15324	15324	0.00%
sqlite	u32_u64_str	unique_0	1024	1046690	1046690	0.00%	1049878	1049878	0.00%
sqlite	u32_u64_str	unique_0	64	74836	74836	0.00%	75828	75828	0.00%

callgrind: serialize_product_value

count	format	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
64	json	47438	47438	0.00%	49988	49988	0.00%
64	bsatn	25716	25716	0.00%	28028	28028	0.00%
16	bsatn	8117	8117	0.00%	9545	9545	0.00%
16	json	12142	12142	0.00%	13944	13944	0.00%

callgrind: update bulk

db	schema	indices	count	preload	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	1024	1024	19944841	19945365	-0.00%	20424407	20425037	-0.00%
stdb_raw	u32_u64_str	unique_0	64	128	1270860	1270707	0.01%	1306460	1306335	0.01%
sqlite	u32_u64_str	unique_0	1024	1024	1802040	1802040	0.00%	1811286	1811286	0.00%
sqlite	u32_u64_str	unique_0	64	128	128386	128386	0.00%	131196	131196	0.00%

On-disk benchmarks

callgrind: empty transaction

db	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	5148	5148	0.00%	5186	5186	0.00%
sqlite	5566	5566	0.00%	6200	6200	0.00%

callgrind: filter

db	schema	indices	count	preload	_column	data_type	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	no_index	64	128	1	u64	77851	77851	0.00%	78303	78303	0.00%
stdb_raw	u32_u64_str	no_index	64	128	2	string	120521	120521	0.00%	121069	121069	0.00%
stdb_raw	u32_u64_str	btree_each_column	64	128	1	u64	23082	23082	0.00%	23490	23490	0.00%
stdb_raw	u32_u64_str	btree_each_column	64	128	2	string	24120	24120	0.00%	24604	24604	0.00%
sqlite	u32_u64_str	no_index	64	128	2	string	145412	145412	0.00%	147374	147370	0.00%
sqlite	u32_u64_str	no_index	64	128	1	u64	124768	124753	0.01%	126498	126483	0.01%
sqlite	u32_u64_str	btree_each_column	64	128	2	string	135412	135412	0.00%	137516	137516	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	1	u64	132245	132245	0.00%	134197	134197	0.00%

callgrind: insert bulk

db	schema	indices	count	preload	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	64	128	816980	817075	-0.01%	867760	867829	-0.01%
stdb_raw	u32_u64_str	btree_each_column	64	128	955910	955558	0.04%	1017888	1017498	0.04%
sqlite	u32_u64_str	unique_0	64	128	413676	413676	0.00%	431082	431082	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	1019717	1019717	0.00%	1059733	1059729	0.00%

callgrind: iterate

db	schema	indices	count	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	1024	151311	151311	0.00%	151345	151345	0.00%
stdb_raw	u32_u64_str	unique_0	64	15296	15296	0.00%	15334	15334	0.00%
sqlite	u32_u64_str	unique_0	1024	1049754	1049754	0.00%	1053636	1053636	0.00%
sqlite	u32_u64_str	unique_0	64	76598	76598	0.00%	77906	77906	0.00%

callgrind: serialize_product_value

count	format	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
64	json	47438	47438	0.00%	49988	49988	0.00%
64	bsatn	25716	25716	0.00%	28028	28028	0.00%
16	bsatn	8117	8117	0.00%	9545	9545	0.00%
16	json	12142	12142	0.00%	13944	13944	0.00%

callgrind: update bulk

db	schema	indices	count	preload	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	1024	1024	18896943	18897157	-0.00%	19455201	19456067	-0.00%
stdb_raw	u32_u64_str	unique_0	64	128	1226645	1226636	0.00%	1291995	1292002	-0.00%
sqlite	u32_u64_str	unique_0	1024	1024	1809601	1809601	0.00%	1818257	1818257	0.00%
sqlite	u32_u64_str	unique_0	64	128	132512	132512	0.00%	135466	135466	0.00%

[github-actions[bot]]

Benchmarking failed. Please check the workflow run for details.

[github-actions[bot]]

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind, an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	5138	5138	0.00%	5176	5176	0.00%
sqlite	5528	5528	0.00%	6026	6026	0.00%

callgrind: filter

db	schema	indices	count	preload	_column	data_type	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	no_index	64	128	1	u64	77841	77841	0.00%	78297	78297	0.00%
stdb_raw	u32_u64_str	no_index	64	128	2	string	120511	120511	0.00%	121127	121127	0.00%
stdb_raw	u32_u64_str	btree_each_column	64	128	1	u64	23072	23072	0.00%	23480	23480	0.00%
stdb_raw	u32_u64_str	btree_each_column	64	128	2	string	24114	24111	0.01%	24602	24599	0.01%
sqlite	u32_u64_str	no_index	64	128	2	string	143491	143491	0.00%	145109	145109	0.00%
sqlite	u32_u64_str	no_index	64	128	1	u64	122842	122842	0.00%	124242	124242	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	2	string	133290	133305	-0.01%	134936	134955	-0.01%
sqlite	u32_u64_str	btree_each_column	64	128	1	u64	130149	130149	0.00%	131639	131639	0.00%

callgrind: insert bulk

db	schema	indices	count	preload	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	64	128	867041	866961	0.01%	886933	886841	0.01%
stdb_raw	u32_u64_str	btree_each_column	64	128	1005808	1008533	-0.27%	1038124	1040641	-0.24%
sqlite	u32_u64_str	unique_0	64	128	396139	396139	0.00%	414231	414227	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	981456	981471	-0.00%	1023028	1023059	-0.00%

callgrind: iterate

db	schema	indices	count	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	1024	151301	151301	0.00%	151343	151343	0.00%
stdb_raw	u32_u64_str	unique_0	64	15286	15286	0.00%	15324	15324	0.00%
sqlite	u32_u64_str	unique_0	1024	1046690	1046690	0.00%	1049878	1049878	0.00%
sqlite	u32_u64_str	unique_0	64	74836	74836	0.00%	75828	75828	0.00%

callgrind: serialize_product_value

count	format	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
64	json	47438	47438	0.00%	49988	49988	0.00%
64	bsatn	25716	25716	0.00%	28028	28028	0.00%
16	bsatn	8117	8117	0.00%	9545	9545	0.00%
16	json	12142	12142	0.00%	13944	13944	0.00%

callgrind: update bulk

db	schema	indices	count	preload	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	1024	1024	19947983	19945365	0.01%	20427141	20425037	0.01%
stdb_raw	u32_u64_str	unique_0	64	128	1270726	1270707	0.00%	1306290	1306335	-0.00%
sqlite	u32_u64_str	unique_0	1024	1024	1802040	1802040	0.00%	1811286	1811286	0.00%
sqlite	u32_u64_str	unique_0	64	128	128386	128386	0.00%	131196	131196	0.00%

On-disk benchmarks

callgrind: empty transaction

db	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	5148	5148	0.00%	5186	5186	0.00%
sqlite	5566	5566	0.00%	6200	6200	0.00%

callgrind: filter

db	schema	indices	count	preload	_column	data_type	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	no_index	64	128	1	u64	77851	77851	0.00%	78303	78303	0.00%
stdb_raw	u32_u64_str	no_index	64	128	2	string	120521	120521	0.00%	121069	121069	0.00%
stdb_raw	u32_u64_str	btree_each_column	64	128	1	u64	23082	23082	0.00%	23490	23490	0.00%
stdb_raw	u32_u64_str	btree_each_column	64	128	2	string	24366	24120	1.02%	24894	24604	1.18%
sqlite	u32_u64_str	no_index	64	128	1	u64	124753	124753	0.00%	126483	126483	0.00%
sqlite	u32_u64_str	no_index	64	128	2	string	145412	145412	0.00%	147374	147370	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	1	u64	132245	132245	0.00%	134197	134197	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	2	string	135412	135412	0.00%	137520	137516	0.00%

callgrind: insert bulk

db	schema	indices	count	preload	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	64	128	817924	817075	0.10%	868678	867829	0.10%
stdb_raw	u32_u64_str	btree_each_column	64	128	956294	955558	0.08%	1017642	1017498	0.01%
sqlite	u32_u64_str	unique_0	64	128	413676	413676	0.00%	431082	431082	0.00%
sqlite	u32_u64_str	btree_each_column	64	128	1019717	1019717	0.00%	1059737	1059729	0.00%

callgrind: iterate

db	schema	indices	count	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	1024	151311	151311	0.00%	151345	151345	0.00%
stdb_raw	u32_u64_str	unique_0	64	15296	15296	0.00%	15334	15334	0.00%
sqlite	u32_u64_str	unique_0	1024	1049754	1049754	0.00%	1053636	1053636	0.00%
sqlite	u32_u64_str	unique_0	64	76598	76598	0.00%	77906	77906	0.00%

callgrind: serialize_product_value

count	format	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
64	json	47438	47438	0.00%	49988	49988	0.00%
64	bsatn	25716	25716	0.00%	28028	28028	0.00%
16	bsatn	8117	8117	0.00%	9545	9545	0.00%
16	json	12142	12142	0.00%	13944	13944	0.00%

callgrind: update bulk

db	schema	indices	count	preload	total reads + writes	old total reads + writes	Δrw	estimated cycles	old estimated cycles	Δcycles
stdb_raw	u32_u64_str	unique_0	1024	1024	18897916	18897157	0.00%	19456284	19456067	0.00%
stdb_raw	u32_u64_str	unique_0	64	128	1226951	1226636	0.03%	1292281	1292002	0.02%
sqlite	u32_u64_str	unique_0	1024	1024	1809601	1809601	0.00%	1818257	1818257	0.00%
sqlite	u32_u64_str	unique_0	64	128	132512	132512	0.00%	135466	135466	0.00%