lemmy icon indicating copy to clipboard operation
lemmy copied to clipboard

Store aggregates without separate tables (e.g. combine `post` and `post_aggregates`)

Open dullbananas opened this issue 1 year ago • 4 comments

This would get rid of a lot of complexity. It would also reduce the chance of things that have been big problems before: trigger bugs, and queries that can't efficiently use indexes.

Diesel's 32 column limit makes this hard. The first thing that should be done is to create a crate with just the schema.rs file, enable the 64 columns feature, apply the changes in schema.rs, and measure the incremental compile time difference using cargo timings. One of these things about the compile time increase caused by the 64 columns feature might be revealed:

  • It's not as bad as before
  • It's outweighed by a compile time decrease caused by the reduced amount of table pairs generated by the allow_tables_to_appear_in_same_query macro

dullbananas avatar Jun 25 '24 20:06 dullbananas

I wouldn't be opposed, but yeah the increase in compile time would be an issue.

You could test the difference by just adding that diesel feature. I believe it'll increase the compile time even if no table has past 32 columns.

dessalines avatar Jul 07 '24 17:07 dessalines

Yes the compile time increases directly after enabling the 64 column feature, no matter if you use them or not. Its because diesel has to generate a lot more code (macros?) to handle these extra columns. So to test the compile time difference, simply add 64 column feature to Lemmy's cargo.toml.

Nutomic avatar Jul 07 '24 22:07 Nutomic

Tested this, the increase is rather minor (2m 41s to 3m 03s with 64 column feature enabled). So we can do this if it gives the advantages you mention.

Nutomic avatar Sep 25 '24 10:09 Nutomic

Nice, in that case I wouldn't be opposed to get rid of all the one-to-one aggregates tables.

dessalines avatar Sep 27 '24 13:09 dessalines