Add indices support (WIP)

[Martoon-00]

Motivation

Currently, there is no way to deal with the database schema purely via Haskell code, which is not good at least because when we define our schema in raw SQL, we get table and fields names defined in several places. I'm planning to make SQL engine know about foreign key constraints in the next PR, and these two seem to provide a basis for moving away from manual schemas definition completely.

PR Details

The following features are implemented:

• [x] Manual indices definition. Here a user can provide a list of indices on the per-table basis, each index is defined by a sequence of its fields and its settings (only most basic settings are supported for now). With syntax as the following.

• [x] Automatic indices detection. A user can make a DatabaseIndices (which is similar to DatabaseSettings but contains information about indices correspondingly) via defaultDbIndices, which gathers indices on those columns which are assumed to participate in JOINs. Later the user can add these indices to the custom ones via dedicated function mergeDbIndices, or immediately use addDefaultDbIndices.

• [x] Manual/automatic migrations support. Previously gathered indices are converted to TableChecks and added to CheckedDatabaseSettings. Two action providers are supplied: ADD INDEX and DROP INDEX. Herewith, unfortunately, there is no SQLite support, since there is no way to modify indices of the existing table there. A common workaround is to rename the table, create a new one with the desired indices and copy all the data, but I'm not yet ready to write copying action providers, although generally, that scheme seems to be perfectly discoverable with existing automatic migration means.

• [ ] Testing.

  • [x] Unit tests on indices definition.
  • [ ] Manual migrations.
  • [ ] Automatic migrations.

• [x] Haddock: builds.

• [ ] Extend the manual.

Notes

I have a set of TODOs remained where I'm not sure about proper design choice, would appreciate hearing your point of view on this.

Also, I added .stylish-Haskell.yaml which corresponds to the style of this project, otherwise I get pretty mad diffs each time I save a file.

[tathougies]

Hello!

Wow.. Thanks! This is a lot to digest, but right off the bat, I'll ask you to rebase on top of the beam-0.8.0.0 branch, since that represents the next release. There'll be a few changes you'll have to make to get it compiling. The upside is that it should trigger a gitlab CI build, which will have to pass before I merge.

Additions to the manual would be nice.

I'm still getting through the code.

Travis

[Martoon-00]

Oh, having CI checking my work would be really nice! I see beam-0.8.0.0-fix-ci branch and multiple travis/beam-0.8.0.0-* braches, which one should I choose?

[tathougies]

Just regular beam-0.8.0.0. I'll get rid of the other ones. They're all merged.

[Martoon-00]

That's weird, I see no plain beam-0.8.0.0 branch :thinking:

[tathougies]

This one: https://github.com/tathougies/beam/tree/travis/beam-0.8.0.0

[Martoon-00]

Ah, thanks. I wasn't sure it was that branch you were talking about because beam-core tests fail to compile there (713522b commit). Could you check?

[tathougies]

The latest commit there should be 377596e

[Martoon-00]

Oh right, I searched in my forked repository. Thanks for helping with figuring this out, I'm rebasing.

[tathougies]

You'll need to resolve conflicts too to get CI working.

[Martoon-00]

Yeah, it's gonna take a while :)

[Martoon-00]

Finally, I rebased upon beam-0.8.0.0 branch. Now there are a couple of small issues:

1. CI here is not triggered, probably because I'm proposing this change from another repo. Is there an option to enable CI in this case and could you do that, or I'd better relocate my code to this repository?
2. I activated CI in my fork for now, and sometimes tests fail https://travis-ci.org/serokell/beam/jobs/470453508. Postgres seems to happily truncate strings after \0 character.

[tp-woven]

Oh, wow, this looks really cool. Any chance of this getting resurrected and merged?

[kmicklas]

@tp-woven You're welcome to take it over! I'm happy to review but unfortunately don't have the time for major feature development on Beam these days.

[Martoon-00]

Yeah, @tp-woven I would be glad if you manage to take this work or pick it as inspiration for your own implementation, especially since @kmicklas approved this. Now I cannot allocate time to this.

If I remember correctly, this PR mostly handles the core part (not tested at all) but may need a decent update of the migration package.

[tp-woven]

Thanks, I'll try to take over. Fair warning: I'm a Haskell noob, so it may take a while... 😅