thunderbird
Test for database upgrades from all releases
In #2158, @philipwhiuk introduced a test for the database upgrade path. This was a great idea! However, it tests only the upgrade path from the early version v29 of the database (k-9 v2.000) to the current version. Missing is a test from new installations of intermediate releases.
I think this is important, since some bugs (e.g. #2103) suggest that there may be more problems (see also comment at https://github.com/k9mail/k-9/pull/2219#issue-206539445), i.e. a difference between create database from scratch in a former release and the upgrade of an older version to that release. People who installed that version as their first version may have still a differing (=potential defective) database today.
Therefore, I propose to write a test, that does:
For every release (from 2.000 to latest; the required list can be generated using the git-tags)
- checkout the respective git-tag
- create a database from scratch using the checked out code (not trivial)
- checkout master (or latest release)
- run upgrade database and compare to freshly created database (like in #2158)
Unfortunately, the second step (create a database from the old version) is not that easy. One problem is that the location of the code for doing that (LocalStore/StoreSchemaDefinition) has changed multiple times in the past. However, I see two possibilities:
a) Execute the original code. Another problem: the constructors of the respective class LocalStore differ in some versions and I don't know if all parameters can be filled with appropriate values in a test environment. Furthermore, the code from different versions has to be executed, which means that the code has to be loaded dynamically.
b) Extract the SQL code from the Java classes and execute it directly. This requires an appropriate parser for analyzing the source code. But this can be done on file basis without executing dynamically loaded code.
What do you think? Any hints on realizing this? I don't know when/if(!) I will have some time for doing this. And I don't know how hard it will be (some details can consume much time, but it could also be done very fast). But I think it's worth it.
Doing this once might be interesting. But I see no value in having automated tests for this. It will only show problems we had in the past. It's not something we can break now if it wasn't broken before, i.e. once the tests pass, they will always pass.
Yes, definitely. It wasn't meant to integrate this into Travis. It would also slow down the tests too much.
However, the test should be automated, because nobody want's to checkout all git-tags manually ... 😉
