couchdb
couchdb copied to clipboard
Query Planner selecting `_all_docs` even though valid indexes exist
Description
Tuesday afternoon, the Query Planner stopped automatically selecting indexes for our _find
queries. This was confirmed by sending the same queries to the _explain
endpoint. This appears to have happened without any any changes to the software using CouchDB or the design documents that should have been used.
To resolve this, we created copies of the design documents, without changing any of the content inside of the document, except for the _id
, appended the suffix "-new". Once doing this, the issue almost immediately resolved itself, and the _explain
endpoint would show that it was using the non-suffixed design document.
Things that we tried, that failed:
- Restarting CouchDB nodes, one at a time
- Restarting microservice accessing CouchDB Cluster
- Restarted virtual machines hosting CouchDB, one at a time
Steps to Reproduce
Issue happened seemingly at random, with no software elease, configuration, or design document changes.
Expected Behaviour
The Query Planner in CouchDB should have selected the proper index, as documented
Your Environment
5 Node CouchDB cluster running in docker containers, deployed via in house Ansible-playbooks. config.ini on each node:
[couch_httpd_auth]
cookie_domain = REDACTED
[couchdb]
uuid = REDACTED
[log]
level = info
[chttpd]
admin_only_all_dbs = true
[smoosh.ratio_dbs]
concurrency= 2
[smoosh.ratio_views]
concurrency= 2
[smoosh.slack_dbs]
concurrency= 2
[smoosh.slack_views]
concurrency= 2
[mango]
default_limit = 1000
[admins]
REDACTED
[couch_httpd_auth]
secret = REDACTED
Cluster Information:
{
"couchdb": "Welcome",
"version": "3.1.1",
"git_sha": "ce596c65d",
"uuid": "1dd6dbe0b018660d2754ad88c2006b67",
"features": [
"search",
"access-ready",
"partitioned",
"pluggable-storage-engines",
"reshard",
"scheduler"
],
"vendor": {
"name": "The Apache Software Foundation"
}
}
Database Information
{
"db_name": "DATABASE_NAME",
"purge_seq": "0-g1AAAAB9eJzLYWBgYMpgTmFQSc4vTc5ISXIoLc4vsiiA8oyM9EqLU_VKk1MK9PJSS3JAqvNYgCRDA5D6DwRZiQx4tBtiak9kSKqH6MsCALbNKe0",
"update_seq": "870085-g1AAAACDeJzLYWBgYMpgTmFQSc4vTc5ISXIoLc4vsiiA8oyM9EqLU_VKk1MK9PJSS3JAqvNYgCRDA5D6DwRZSQxsy9_jMcEQ04REhqR6qNZ517IA2sctBQ",
"sizes": {
"file": 895381990,
"external": 1838490411,
"active": 875249258
},
"props": {},
"doc_del_count": 455,
"doc_count": 379055,
"disk_format_version": 8,
"compact_running": false,
"cluster": {
"q": 2,
"n": 3,
"w": 2,
"r": 2
},
"instance_start_time": "0"
}
- CouchDB version used: 3.1.1, using the
ibmcom/couchdb3
image on DockerHub - Browser name and version: N/A, access is done over NodeJS HTTP Client Libraries
- Operating system and version: Red Hat Enterprise Linux 8.3
Additional Context
When using the use_index
parameter, it worked without issues.
In the documentation, it does state that "It’s good practice to specify indexes explicitly in your queries". This is something we're looking to get situated, though many of our microservices rely on the Query Planner to pre-select their index.