mongoengine Cannot save documents with dots in a field key.

Cannot save documents with dots in a field key.

Open benjhastings opened this issue 3 years ago • 4 comments

Issue https://github.com/MongoEngine/mongoengine/issues/1594 (related: https://github.com/MongoEngine/mongoengine/pull/2193) suggests that it has resolved the issue with having dots in a key name. However, this does't seem to be the case. Tested with mongoengine==0.22.1 and pymongo==3.11.2.

Although I'm raising it here I think that this is most likely a limitation of pymongo itself as https://docs.mongodb.com/manual/core/document/#field-names suggests that dots in key names aren't supported by official MongoDB drivers. However, I was just hoping to get a sanity check that https://github.com/MongoEngine/mongoengine/issues/1594 / https://github.com/MongoEngine/mongoengine/pull/2193 don't actually resolve the problem. I feel like in fields#DictField.validate that key_has_dot_or_dollar should be added back in (or possibly just a modified key_has_dot helper function)

For confirmation, I've tested with a $ in the key name and this works - it's just . that doesn't work.

A simple test case

import mongoengine
from mongoengine import fields, Document

mongoengine.connect(db="my_test_db")

class DocumentWithDots(Document):
    data = fields.DictField()

if __name__ == '__main__':
    DocumentWithDots(data={"t.est": "data"}).save()

Expected

document to be saved without issue

Actual

Traceback (most recent call last):
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/mongodot.py", line 14, in <module>
    DocumentWithDots(data={"t.est": "data"}).save()
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/mongoengine/document.py", line 398, in save
    object_id = self._save_create(doc, force_insert, write_concern)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/mongoengine/document.py", line 463, in _save_create
    object_id = wc_collection.insert_one(doc).inserted_id
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/collection.py", line 701, in insert_one
    session=session),
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/collection.py", line 615, in _insert
    bypass_doc_val, session)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/collection.py", line 603, in _insert_one
    acknowledged, _insert_command, session)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1498, in _retryable_write
    return self._retry_with_session(retryable, func, s, None)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1384, in _retry_with_session
    return self._retry_internal(retryable, func, session, bulk)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1416, in _retry_internal
    return func(session, sock_info, retryable)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/collection.py", line 598, in _insert_command
    retryable_write=retryable_write)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/pool.py", line 699, in command
    self._raise_connection_failure(error)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/pool.py", line 694, in command
    exhaust_allowed=exhaust_allowed)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/network.py", line 122, in command
    codec_options, ctx=compression_ctx)
  File "/home/ben/.virtualenvs/tmp-86f966e38f49f679/lib/python3.6/site-packages/pymongo/message.py", line 715, in _op_msg
    flags, command, identifier, docs, check_keys, opts)
bson.errors.InvalidDocument: key 't.est' must not contain '.'

Process finished with exit code 1