redash
redash copied to clipboard
getredash
AWS DocumentDB Support
- Redash Version: Version: 8.0.0+b32245 (a16f551e)
- Browser/OS: Chrome / MacOS 84.0.4147.105
- How did you install Redash: Self hosted AWS AMI (ami-0b97435028ca44fcc)
As it currently stands Redash does not work with AWS DocumentDB. When you go to create your first query, it breaks with Schema refresh failed. This issue is also being talked about in the Discuss community at: https://discuss.redash.io/t/schema-refresh-failed-mongodb-datasource/4020
We know the Python source code is breaking in:
https://github.com/getredash/redash/blob/ecb9adf903842694ab01479727709064296ea029/redash/query_runner/mongodb.py#L229-L241
I am not a Python developer (Node.js) and nor do I have a Python environment setup. However, I am willing to provision a AWS DocumentDB and provide credentials and access temporarily to anybody willing to debug and work on this issue.
@nodesocket
I have some clues on this problem.
The _get_collection_fields function is called when Redash refreshes the data schema
https://github.com/getredash/redash/blob/ecb9adf903842694ab01479727709064296ea029/redash/query_runner/mongodb.py#L218-L220
which makes use of $natural operator for sorting.
However, when referring to the latest developer guide of Amazon DocumentDB
I found that DocumentDB doesn't support $natural operator.
It would be nice if you can share / grant an access temporarily for me to test on a development database. In a long run, probably, it is best to create a separate query runner to support DocumentDB.
@tim5go thanks so much for following up. Indeed AWS DocumentDB unfortunately does not support all the MongoDB aggregate functions (though they are slowly adding support).
Do you have any ideas if most of the functionality of Redash would even work beyond listing the collections without all supported MongoDB aggregate functions?
I am happy to provide you with a test/playground DocumentDB instance as well. Can you send me an e-mail at justin @ my githubusername dot com?
Anybody; prefer core Redash developers, willing to take a look at adding DocumentDB as an officially supported data source? AWS recently upgraded DocumentDB to version 4.0 [1] (still don't think it supports the required MongoDB operator $natural).
[1] https://aws.amazon.com/blogs/database/introducing-amazon-documentdb-with-mongodb-compatibility-4-0/
Compatibility for $natural has been added
https://www.amazonaws.cn/en/new/2021/amazon-documentdb-mongodb-compatibility-adds-support-and-indexing-improvements/
I tried building a separate image for deployment for AWS DocumentDB. You can replace redash's image with this image: double2d/redash-docdb:0.1 or double2d/redash-docdb:latest. I used this image in EC2 connecting to my DocDB using the connection string provided by AWS and everything works fine.
