Text search raises _Cursor__killed AttributeError

[enugentdt]

Hello,

I've been trying to perform text searching in umongo, and have come across the following error:

Exception ignored in: <bound method Cursor.__del__ of <pymongo.cursor.Cursor object at 0x10b43df98>>
Traceback (most recent call last):
  File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/pymongo/cursor.py", line 240, in __del__
    self.__die()
  File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/pymongo/cursor.py", line 297, in __die
    already_killed = self.__killed
AttributeError: 'Cursor' object has no attribute '_Cursor__killed'
Traceback (most recent call last):
  File "project.py", line 95, in <module>
    app = asyncio.get_event_loop().run_until_complete(application_factory())
  File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
    return future.result()
  File "project.py", line 64, in application_factory
    await cloud_db.search_for_posts('ovirt')
  File "/Users/space/Documents/Programming/project/frontend-server/db/cloud_db.py", line 264, in search_for_posts
    {'score': {'$meta': 'textScore'}}).sort([('score', {'$meta': 'textScore'})])
  File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/umongo/frameworks/motor_asyncio.py", line 247, in find
    return WrappedCursor(cls, cls.collection.find(*args, filter=filter, **kwargs))
  File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/motor/core.py", line 567, in find
    **unwrap_kwargs_session(kwargs))
  File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/pymongo/collection.py", line 1445, in find
    return Cursor(self, *args, **kwargs)
TypeError: __init__() got multiple values for argument 'filter'

Minimum code to cause an error:

docs = Document.find(
        {'$text': {'$search': 'some words'}},
        {'score': {'$meta': 'textScore'}}).sort([('score', {'$meta': 'textScore'})])
    return docs

where Document is a random document that you can create with a text index. This code is copied from the pymongo sample code, since there was no documentation for this on the RTD. Am I being dumb about something (wouldn't be the first time today)?

Link where the pymongo code was found:

http://api.mongodb.com/python/current/api/pymongo/cursor.html#pymongo.cursor.Cursor.sort

Thanks!

[touilleMan]

Hi,

Have you tried to pass the find parameters as kwargs ?

Document.find(
    filter={'$text': {'$search': 'some words'}},
    projection={'score': {'$meta': 'textScore'}}
).sort([('score', {'$meta': 'textScore'})])

[enugentdt]

I've been trying that, but I get a KeyError: Mapping Key Not Found on the text score

[touilleMan]

Hi,

I've done so test on your issue:

from pymongo import MongoClient
from umongo import Instance, Document, fields, validate

db = MongoClient().test
db.foo.create_index([('bar', 'text')])

db.foo.insert({'bar': 'Hello John'})
db.foo.insert({'bar': 'Hello Philippe'})
db.foo.insert({'bar': 'Bye Philippe'})

instance = Instance(db)
@instance.register
class Foo(Document):
    bar = fields.StringField()

docs = Foo.find(
    filter={'$text': {'$search': 'hello'}},
    projection={'score': {'$meta': 'textScore'}}
).sort([('score', {'$meta': 'textScore'})])

print(list(docs))

result:

$ python test-142.py 
Traceback (most recent call last):
  File "/home/emmanuel/projects/umongo/umongo/data_proxy.py", line 68, in from_mongo
    field = self._fields_from_mongo_key[k]
KeyError: 'score'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test-142.py", line 23, in <module>
    print(list(docs))
  File "/home/emmanuel/projects/umongo/umongo/frameworks/pymongo.py", line 49, in __iter__
    yield self.document_cls.build_from_mongo(elem, use_cls=True)
  File "/home/emmanuel/projects/umongo/umongo/document.py", line 205, in build_from_mongo
    doc.from_mongo(data, partial=partial)
  File "/home/emmanuel/projects/umongo/umongo/document.py", line 215, in from_mongo
    self._data.from_mongo(data, partial=partial)
  File "/home/emmanuel/projects/umongo/umongo/data_proxy.py", line 72, in from_mongo
    .format(key=k, cls=self.__class__.__name__)))
umongo.exceptions.UnknownFieldInDBError: FooDataProxy: unknown "score" field found in DB.

The error is due to the projection field: this change the format of returned data so umongo doesn't recognize the document it is supposed do deserialize !

From this what you can do:

• correct your request not to use the projection field: Document.find(filter={'$text': {'$search': 'some words'}}). This works fine, but it's not the request you were looking to do :'-(
• use pymongo for this request, then manually remove the score field from the data and deserialize them using the Document.build_from_mongo method

Finally I'm considering to add a check in the umongo find API to forbid using the projection param, explaining the issue about it. What do you think ?

[enugentdt]

For anybody finding this issue:

I found that an aggregate pipeline works. Here's my code.

foos = db.foo.aggregate([
        {"$match": {"$text": {"$search": text}}},
        {"$sort": {"score": {"$meta": "textScore"}}},
        {"$project": {"score": 1, "_id": 0}} # Fields you want
    ])

for foo in await foos.to_list(length=limit):
    # Code with a foo from foos

Personally, the most important thing to me is that the feature exists. I don't know what the correct solution is for ensuring that, though. I assume it's near impossible to make the projection field work correctly?

If nothing else, a warning/error would be good - that way we don't have undefined behavior.

[dmnorc]

Also you can do cursor = Foo.collection.find(filter, projection=projection)