couchdb icon indicating copy to clipboard operation
couchdb copied to clipboard

Query all document (_all_docs) from specific partition with filtering returns wrong total_rows value and document information

Open vmatyus opened this issue 5 years ago • 3 comments

Description

During the usage of POST /{db}/_partition/{partition_key}/_all_docs in the response there is a field total_rows and its value should return the number of documents in the shard range hosting that partition. Furthermore it should return only partition specific documents.

If there is keys filter included in the query, currently the request returns the number of documents in the database and all partition related document that matches with the filter. There is no partition filtering.

Steps to Reproduce

  1. Create a partitioned database
  2. Create sample partitioned documents
  3. Query the database with endpoint: POST /{db}/_partition/{partition_key}/_all_docs --data-raw {} It shall return a total_rows value. Let's call it n. Returns all partition specific document.
  4. Query the database with an additional filter. eg: POST /{db}/_partition/part2/_all_docs --data-raw {"keys":["part1:doc1", "part2:doc1"]} In this case the total_rows of the response contains the number of all document of the database .

In the request the following can be seen:

"rows": [
        {
            "id": "part1:doc1",
            "key": "part1:doc1",
            "value": {
                "rev": "5-aa9d2b82de6db1e61fba8e4057299d21"
            }
        },
        {
            "id": "part2:doc1",
            "key": "part2:doc1",
            "value": {
                "rev": "3-a707bb62c86bd430389e4d596aeb6084"
            }
        }
]

Expected Behaviour

The total_rows field should have value n. Only partition specific document information should be returned. This should be the expected response:

"rows": [
        {
            "key": "part1:doc1",
            "error": "not_found"
        },
        "id": "part2:doc1",
            "key": "part2:doc1",
            "value": {
                "rev": "3-a707bb62c86bd430389e4d596aeb6084"
            }
    ]

Your Environment

  • CouchDB version used: CouchDB 3.0.0-6d432a9
  • Browser name and version: PostMan 7.15.0
  • Operating system and version: MacOS desktop Catalina 10.15.2

Additional Context

vmatyus avatar Jan 28 '20 16:01 vmatyus

The wrong number for total_rows is also returned when using a view:

curl -b AuthSession=my-session 127.0.0.1:5984/my_db/_partition/my_partition          
{
    "db_name": "my_db",
    "sizes": {
        "active": 996092,
        "external": 1369131
    },
    "partition": "my_partition",
    "doc_count": 1264,
    "doc_del_count": 6
}

curl -b AuthSession=my-session 127.0.0.1:5984/my_db/_partition/my_partition/_design/my_design/_view/my_view
{
    "total_rows": 2348,
    "offset": 0,
    "rows": [
        // ...
    ]
}

When using a view the total_rows shows data for the entire database not just for the partition.

David-Klemenc avatar Jul 27 '20 06:07 David-Klemenc

Bumping this issue, still exists in 8/12/2022. Any direction on where to start working on this issue? Maybe I can help?

zachlankton avatar Aug 12 '22 13:08 zachlankton

Bumping this issue, still exists in 8/12/2022. Any direction on where to start working on this issue? Maybe I can help?

Hi, maybe you can start investigating the issue by following the partition request.

big-r81 avatar Aug 13 '22 12:08 big-r81