Extend support to multiple database technologies

[evilaliv3]

Storm time is coming to the end.

Possible adopters expressed the need to have support for SQL Server and day by day Storm is more and more unmainatined.

We should start discussing for a possible replacement;

requirements for the new library to be chosen are:

• good community
• multiiple db support (storm not support sqlserver nor oracle!)

If we where to change the library of the project (twisted, and this is not the case) my preference would go for Django and it's ORM. given that at this change would not be possible i think the only valid alternative is SQLAlchemy: http://www.sqlalchemy.org/

SQAlchemy seems to benefit of a wide community righ now and supports the following db in the core:

• Firebird
• Microsoft SQL Server
• MySQL
• Oracle
• PostgreSQL
• SQLite
• Sybase

In addition it has additional support by third party additions.

[fpietrosanti]

@evilaliv3 we already evaluated sqlalchemy at the beginning of GlobaLeaks 2 rewrite but it doesn't seems to be usable in any way with Twisted. I suggest to first look if there is a software using it sqlalchemy with twisted and how.

Which is the actual limitation of storm?

[evilaliv3]

1. for twisted there is a specific patch in order to make possible using sqlalchemy; obviously we should test it integrating the tests of sqlalchemy and validating the solution for all the main db befor staring changing the technology

2. the main limistation is the support for only sqlite, mysql and postrpgres. nor sqlserver nor oracle.

[fpietrosanti]

@evilaliv3

1. Are there any software publicly using this twisted/sqlalchemy patch to evaluate the robustness of the solution?

2. So, the only requirement currently is if we will need to support SQLServer for ANAC right? In that case, we should technically analyze separately the status/opportunity/cost of supporting SQLServer in Storm (https://bugs.launchpad.net/storm/+bug/145317)

[evilaliv3]

Following the adoption of SQLAlchemy the following databases are now supported:

• Firebird
• Microsoft SQL Server
• MySQL
• Oracle
• PostgreSQL
• SQLite
• Sybase

Up to date list of supported databases at: http://docs.sqlalchemy.org/en/latest/dialects/index.html

While implementing the migration to SQLALchemy i took personally care of testing Models creation on Microsoft SQL Server, MySQL, PostgreSQL

it will be worth experimenting with: making possible for the application to configure the database link during the wizard

\cc @fpietrosanti @ncommander It would be nice as well to experimenting using an in memory database until the wizard is completed.

[fspegni]

Is there any documentation on how to configure SQLAlchemy in GL to use PostgreSQL or MySQL? Thanks

[evilaliv3]

Thank you for your question @fspegni

Actually the current status GlobaLeaks is that is support SQLAlchemy that would enable (in theory) the possibility of using an external database alternative to sqlite but making globaleaks work with an external database would require major work and testing.

I've to say that by the current research on this topic, we consider that extending globaleaks to use an external database would lead the application out of its current threat model definition that consider the application to be self-contained and sandboxed to limit possible mistakes and reduce the software exposure.

If you could detail better your needs i may advice on alternatives. I invite you as well to join our user community on slack at: https://slack.globaleaks.org.