wekan
Question: back end other than Mongo?
From @romu70 on November 16, 2016 10:11
Hi, Is there any simple way to setup WeKan with another back end than Mongo? Is this on the roadmap?
Thanks.
Copied from original issue: wefork/wekan#9
@romu70
Could you do some Google searching for projects that make interface like Mongo but translate it to another database backend like PostgreSQL or MySQL ? And also find examples how Meteor / Mongo projects have added support for other databases?
@romu70
Somebody else did ask about this before, so I added it to roadmap. I have not had time to add all feature requests from original wekan project to roadmap yet.
From @romu70 on November 16, 2016 10:58
Thanks a lot, so I think you deserve a little explanation ;-)
Self hosting is great but raises some issues, even more if you plan to do self-hosting with Docker:
- the more is the list of apps you want to host, the more is the list of required data back ends, and this makes self hosting more and more difficult, and resources consuming, both human and hardware
- when you self-host an application through Docker, you absolutely need to externalize the data if you want to be able to update the Docker without loosing things.
Personally, I don't need specifically a SQL DB, Redis would be great also, and maybe easier for switching from a NoSQL DB, but I don't know well both.
Thanks a lot.
From @pierreozoux on November 17, 2016 10:18
Meteor is using Mongo. Wekan uses Meteor as a Framework. As long as Meteor doesn't allow other db, wekan can't use any other db.
Closing this for now.
Feel free to reopen once meteor add support to other db.
@pierreozoux
There can be some way to use Meteor with other databases, I did see some search results about that, it's not impossible. This is looking for somebody needing other database support to do the research.
Reopening issue.
From @aegrey on December 18, 2016 2:34
Meteor said last year it was a high priority, but with MongoDB's growing popularity I would assume it's become less so.
There's a library that does it (numtel:mysql - haven't used it in a while). Hasn't been updated since mid-2016, but his example project is active (https://github.com/numtel/meteor-mysql-leaderboard). For those interested, you could try integration in your fork (I am just delving into Wekan's code, so couldn't say how heavy of a lift it'd be).
Currently Wekan works like this:
Frontend Javascript <=> Backend Node.js <=> MongoDB
Is there this kind of Database Query converter somewhere?
Frontend Javascript <=> Backend Node.js <=> Database Query converter <=> MongoDB/MariaDB/PostgreSQL etc
When Node.js sends MongoDB query, Database Query converter would then use something like search-replace/regex/parsing to convert it to MariaDB syntax, get results from MariaDB and send them in MongoDB format back to Node.js.
This Database Query converter would have it's own web interface, where database queries can be translated to different database syntax, storing those at converter's own syntax database. If some syntax was not able to be converted, there would be error listed at error log page of converter's own web interface.
This way we could use any database, so if there is already some existing database then it's not required to install MongoDB, just use existing database.
That Database Query converter could be also used for auditing, stats, logging, permission management, limiting to only allowed queries, exposing REST API to other software etc.
It's quite possible. IMO, you need to have the following changes,
- Building a DB query converter that plays along meteor.
- Changing the whole codebase that calls MongoDB in any way.
- Making unit tests to make sure nothing falls out and the performance does not drop.
If not from scratch, then we could try to use some other libraries available online to avoid repetition.
- https://www.npmjs.com/package/mongo2mysql
- https://www.npmjs.com/package/mongo-mysql
- https://github.com/doubaokun/MongoDB-to-MySQL
- https://www.npmjs.com/package/mysql-mongo-db
- https://www.npmjs.com/package/fruit
- https://www.npmjs.com/package/db-migrate
- https://www.npmjs.com/package/databridge
- https://www.npmjs.com/package/east
We can try to find if this is possible or not.
I do not mean changing anything in Wekan code.
I mean replacing MongoDB with, for example, single executeable binary:
- It works in same port as MongoDB, port could be changed in environment variable:
- That binary could be programmed with any language, like Go
- The same binary could also include database server that's made with Go
- When receiving MongoDB query, it could use internal database server, or convert query to other database syntax, connect to that database, get results, convert results to MongoDB format, and return to that app that uses MongoDB. I mean any app that uses MongoDB, not just Wekan.
I'm currently looking at ToroDB:
- https://www.torodb.com
- https://github.com/torodb/torodb
- https://twit.tv/shows/floss-weekly/episodes/377
Ok I got ToroDB working on docker-compose, so this it Wekan reporting with new LibreOffice 3.5 native PostgreSQL support, using Wekan <=> MongoDB <=> ToroDB => PostgreSQL read-only mirroring. I will create new repo wekan-postgresql for this. You can use any programming language, Office suite or reporting package to get read-write MongoDB data and read-only PostgreSQL data with SQL queries.
Ok I have now added docker-compose.yml etc code to: https://github.com/wekan/wekan-postgresql
At ToroDB website was said, that it could be possible to use only ToroDB/PostgreSQL and not write data to MongoDB at all, completely replacing MongoDB, but I did't yet figure out how to do it.
Hi there,
A ToroDB developer here! As you may know, ToroDB Stampede is on beta right now, in fact, we are looking for real applications to find (and solve!) possible incompatibilities and we would be glad to help if you find problems. On the other hand, Stampede has been designed to read from a MongoDB. We have another product called ToroDB Server which can be used instead of MongoDB (talking the same protocol), but it is on developer preview version and Meteor is not supported (as ToroDB Server does not support replication yet).
\cc @ahachete
As @gortiz said, ToroDB Stampede (https://www.torodb.com/stampede/) replicates from MongoDB and will provide a read-only, native SQL version of your MongoDB data in real-time.
If you want to completely replace MongoDB, you will need to wait a bit for ToroDB Server (https://www.torodb.com/ "ToroDB Technology") to be available (ETA: Q3 2017).
We would be happy to help out understand your requirements and how ToroDB may fit in your stack. Please let us know :)
@gortiz @ahachete
Some time ago I tried wekan-postgresql and it had errors starting, has something changed in docker-compose file? https://github.com/wekan/wekan-postgresql
Is it possible to test Wekan with ToroDB server?
@gortiz @ahachete
When working with Node.js applications, have you come across errors like at #1096 ? If yes, do you have ideas how to debug and fix ?
Not sure if you guys noticed, mysql has had native support for json documents for over a year now also including a REST api built in into the database. IMO picking up support for mysql could be relatively easy + give anyone the luxury of combining things like joins, stored procedures and the whole sql thing including replication with the flexibility of a document store. Oh and performance is really good too.
@killua-eu
Are there Meteor.js projects that use JSON support or REST API in MySQL ?
What does "relatively easy" mean in practice? How to convert Meteor.js project to use MySQL ?
@xet7 , that I won't be able to answer well with respect to Meteor.js - javascript is definitely not my strong point. Nontheless I've been using the mysql xdevapi and the builtin sql functions to manipulate and query json documents since the feature was available with python and php - never looked back. Js has been a first class citizen here with https://devhub.io/repos/mysql-mysql-connector-nodejs appearing right after the feature has been made available.
I made little research: https://stackoverflow.com/a/20616188/460695 recommends https://github.com/mysqljs/mysql (the link in the post is different but doesn't error out, which has to mean the original was renamed) for complete support (looking at the source code it has a wide array of features, so I can see why it's recommended) in pure Node.js but doesn't have reactive SELECT, for which some place recommended https://github.com/numtel/meteor-mysql.
Seems, with those you're most of the time using functions instead of writing MySQL yourself. However, should you have to and need help, I'm available. I'm not awesome at it (or SQLite 3, which is largely similar) but not trash either.
@rautamiekka
Database abstraction layer by using functions would make supporting more databases much easier.
Is there documentation how to port Meteor.js/MongoDB application like Wekan to it?
@rautamiekka
If you have time to try even beginning steps of implementation to see what is actually required, it would be nice.
There is no requirements for spending a long time or being awesome at it.
^ Meant MySQL code that's known to be needed, and is known what it should do.
Is there any update? I would like to use Wekan with MariaDB or mysql, due my hosting provider (VPS), doesnt allow a easy way to let it run mongodb :/
Does somebody know would it be possible to use this database orm with Meteor.js app like Wekan? https://www.npmjs.com/package/typeorm
IMO Rethink is best suited for real-time apps such as chat. Has also some limitations https://rethinkdb.com/limitations/ i.e. with regards to unicode collations. While I still think Rethink is a better database then Mongo (https://rethinkdb.com/docs/comparison-tables/) I'd still target Mysql (even over postgres) - has all the features of a fully fledged sql database, can do json, has a CRUD api, does replication well, has pain-free upgrade paths (unlike most other databases where version bumps are a real PITA).
SQLite should also be considered. Simplest option - runs everywhere, zero setup.
A good idea is to copy various other services (e.g. gogs/gitea) - out of the box, sqlite is supported. But if you want something more "serious" then you need to add mysql/postgres/mongo.
