Read-only database user can not query documents on first attempt

[markf94]

A database created with mongoengine is unusable with a read-only user since they cannot create indexes in the database. This seems like a bug and not a feature since a read-only user should never be expected to create indices in the first place.

Code example

# models.py

class TestModel(Document):
    some_field = StringField(unique=True)

Now assume that we initialized the database with a read-write user with some TestModel documents. This also created the required index on some_field.

Let's now, switch to a read-only user and run this script:

from .models import TestModel

try:
    first_model = TestModel.objects.first()  # will fail (see error message below)
except pymongo.errors.OperationFailure:
    first_model = TestModel.objects.first()  # will succeed

The problem here is that the first time TestModel.objects.first() results in something similar to that:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib/python3.9/site-packages/mongoengine/queryset/manager.py", line 37, in __get__
    queryset = queryset_class(owner, owner._get_collection())
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib/python3.9/site-packages/mongoengine/document.py", line 215, in _get_collection
    cls.ensure_indexes()
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib/python3.9/site-packages/mongoengine/document.py", line 894, in ensure_indexes
    collection.create_index(fields, background=background, **opts)
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib64/python3.9/site-packages/pymongo/collection.py", line 2059, in create_index
    return self.__create_indexes([index], session, **cmd_options)[0]
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib64/python3.9/site-packages/pymongo/collection.py", line 1949, in __create_indexes
    self._command(
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib64/python3.9/site-packages/pymongo/collection.py", line 238, in _command
    return sock_info.command(
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib64/python3.9/site-packages/pymongo/pool.py", line 683, in command
    return command(self, dbname, spec, slave_ok,
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib64/python3.9/site-packages/pymongo/network.py", line 159, in command
    helpers._check_command_response(
  File "/home/markf94/.cache/pypoetry/virtualenvs/warehouse-cHYSmYzu-py3.9/lib64/python3.9/site-packages/pymongo/helpers.py", line 164, in _check_command_response
    raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: not authorized on warehouse to execute command { createIndexes: "test_model", indexes: [ { background: false, unique: true, sparse: false, name: "some_field_1", key: { asset_id: 1 } } ], lsid: { id: UUID("df8eb5c2-da80-4000-817c-02aba7eee0d0") }, $clusterTime: { clusterTime: Timestamp(1618519417, 1), signature: { hash: BinData(0, 69DC82B4E30F615530E0294F94AEE7397CB376EE), keyId: 6951022745983385602 } }, $db: "warehouse", $readPreference: { mode: "primary" } }, full error: {'operationTime': Timestamp(1618519417, 1), 'ok': 0.0, 'errmsg': 'not authorized on warehouse to execute command { createIndexes: "test_model", indexes: [ { background: false, unique: true, sparse: false, name: "some_field_1", key: { some_field: 1 } } ], lsid: { id: UUID("df8eb5c2-da80-4000-817c-02aba7eee0d0") }, $clusterTime: { clusterTime: Timestamp(1618519417, 1), signature: { hash: BinData(0, 69DC82B4E30F615530E0294F94AEE7397CB376EE), keyId: 6951022745983385602 } }, $db: "warehouse", $readPreference: { mode: "primary" } }', 'code': 13, 'codeName': 'Unauthorized', '$clusterTime': {'clusterTime': Timestamp(1618519417, 1), 'signature': {'hash': b'i\xdc\x82\xb4\xe3\x0faU0\xe0)O\x94\xae\xe79|\xb3v\xee', 'keyId': 6951022745983385602}}}

whereas the second time running it works just fine. I am aware that I can set auto_create_index to False in meta but that isn't the solution I'm looking for because I still want automatic index creation. However, once the indices are there, there should be no need to recreate them especially not with a read-only user.

Isn't there a way that cls.ensure_indexes() in mongoengine/document.py could first check for the existing indices and only try and create them if they don't exist? Or just skip the call to cls.ensure_indexes() alltogether if the user is read-only?

[markf94]

I feel like we could use compare_indexes() to fix this issue since it compares the required indexes with the existing indexes.

[bagerard]

HI @markf94, this would be a great addition, would you have time to work on this by any chance?

[markf94]

@bagerard yeah I would be keen to contribute that but I'm currently really swamped at work. Hoping to get to it within the next month!

[bagerard]

Sounds good :+1:

[acbergan]

Any movement on this? I would benefit from a fix.