umongo
umongo copied to clipboard
Text search raises _Cursor__killed AttributeError
Hello,
I've been trying to perform text searching in umongo, and have come across the following error:
Exception ignored in: <bound method Cursor.__del__ of <pymongo.cursor.Cursor object at 0x10b43df98>>
Traceback (most recent call last):
File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/pymongo/cursor.py", line 240, in __del__
self.__die()
File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/pymongo/cursor.py", line 297, in __die
already_killed = self.__killed
AttributeError: 'Cursor' object has no attribute '_Cursor__killed'
Traceback (most recent call last):
File "project.py", line 95, in <module>
app = asyncio.get_event_loop().run_until_complete(application_factory())
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
return future.result()
File "project.py", line 64, in application_factory
await cloud_db.search_for_posts('ovirt')
File "/Users/space/Documents/Programming/project/frontend-server/db/cloud_db.py", line 264, in search_for_posts
{'score': {'$meta': 'textScore'}}).sort([('score', {'$meta': 'textScore'})])
File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/umongo/frameworks/motor_asyncio.py", line 247, in find
return WrappedCursor(cls, cls.collection.find(*args, filter=filter, **kwargs))
File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/motor/core.py", line 567, in find
**unwrap_kwargs_session(kwargs))
File "/var/root/.local/share/virtualenvs/frontend-server-bGNHtS7t/lib/python3.6/site-packages/pymongo/collection.py", line 1445, in find
return Cursor(self, *args, **kwargs)
TypeError: __init__() got multiple values for argument 'filter'
Minimum code to cause an error:
docs = Document.find(
{'$text': {'$search': 'some words'}},
{'score': {'$meta': 'textScore'}}).sort([('score', {'$meta': 'textScore'})])
return docs
where Document is a random document that you can create with a text index. This code is copied from the pymongo sample code, since there was no documentation for this on the RTD. Am I being dumb about something (wouldn't be the first time today)?
Link where the pymongo code was found:
http://api.mongodb.com/python/current/api/pymongo/cursor.html#pymongo.cursor.Cursor.sort
Thanks!
Hi,
Have you tried to pass the find
parameters as kwargs ?
Document.find(
filter={'$text': {'$search': 'some words'}},
projection={'score': {'$meta': 'textScore'}}
).sort([('score', {'$meta': 'textScore'})])
I've been trying that, but I get a KeyError:
Mapping Key Not Found
on the text score
Hi,
I've done so test on your issue:
from pymongo import MongoClient
from umongo import Instance, Document, fields, validate
db = MongoClient().test
db.foo.create_index([('bar', 'text')])
db.foo.insert({'bar': 'Hello John'})
db.foo.insert({'bar': 'Hello Philippe'})
db.foo.insert({'bar': 'Bye Philippe'})
instance = Instance(db)
@instance.register
class Foo(Document):
bar = fields.StringField()
docs = Foo.find(
filter={'$text': {'$search': 'hello'}},
projection={'score': {'$meta': 'textScore'}}
).sort([('score', {'$meta': 'textScore'})])
print(list(docs))
result:
$ python test-142.py
Traceback (most recent call last):
File "/home/emmanuel/projects/umongo/umongo/data_proxy.py", line 68, in from_mongo
field = self._fields_from_mongo_key[k]
KeyError: 'score'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test-142.py", line 23, in <module>
print(list(docs))
File "/home/emmanuel/projects/umongo/umongo/frameworks/pymongo.py", line 49, in __iter__
yield self.document_cls.build_from_mongo(elem, use_cls=True)
File "/home/emmanuel/projects/umongo/umongo/document.py", line 205, in build_from_mongo
doc.from_mongo(data, partial=partial)
File "/home/emmanuel/projects/umongo/umongo/document.py", line 215, in from_mongo
self._data.from_mongo(data, partial=partial)
File "/home/emmanuel/projects/umongo/umongo/data_proxy.py", line 72, in from_mongo
.format(key=k, cls=self.__class__.__name__)))
umongo.exceptions.UnknownFieldInDBError: FooDataProxy: unknown "score" field found in DB.
The error is due to the projection field: this change the format of returned data so umongo doesn't recognize the document it is supposed do deserialize !
From this what you can do:
- correct your request not to use the projection field:
Document.find(filter={'$text': {'$search': 'some words'}})
. This works fine, but it's not the request you were looking to do :'-( - use pymongo for this request, then manually remove the
score
field from the data and deserialize them using theDocument.build_from_mongo
method
Finally I'm considering to add a check in the umongo find
API to forbid using the projection
param, explaining the issue about it. What do you think ?
For anybody finding this issue:
I found that an aggregate pipeline works. Here's my code.
foos = db.foo.aggregate([
{"$match": {"$text": {"$search": text}}},
{"$sort": {"score": {"$meta": "textScore"}}},
{"$project": {"score": 1, "_id": 0}} # Fields you want
])
for foo in await foos.to_list(length=limit):
# Code with a foo from foos
Personally, the most important thing to me is that the feature exists. I don't know what the correct solution is for ensuring that, though. I assume it's near impossible to make the projection field work correctly?
If nothing else, a warning/error would be good - that way we don't have undefined behavior.
Also you can do
cursor = Foo.collection.find(filter, projection=projection)