elasticsearch-dsl-py icon indicating copy to clipboard operation
elasticsearch-dsl-py copied to clipboard

Add async compatible classes

Open sethmlarson opened this issue 3 years ago • 15 comments

Closes #921 Closes #1355

TODOs:

  • [ ] Figure out Mapping functions that use I/O, my first thought is to deprecate them in 7.x and remove in 8.x and instead use the Mapping class as only for storing metadata. These functions don't appear to be used anywhere in the codebase so shouldn't have a big impact on users?
  • [ ] See if we can leverage unasync for unit tests as well? Either way need to test the new Async classes
  • [ ] Add section of documentation on Async variants of Search, etc
  • [ ] Document that AsyncSearch doesn't have anything to do with the /_async_search/* endpoints
  • [ ] Add section in contribution documentation on the meaning of _async/, _base, and _sync directories
  • [ ] Try to move all the imports to the top of files, in theory this should be possible now because of _base
  • [x] ~Create an example FastAPI application with AsyncDocument and AsyncFacetedSearch~ (captured in issue https://github.com/elastic/elasticsearch-dsl-py/issues/1482)

sethmlarson avatar Jan 14 '21 21:01 sethmlarson

With pull request sethmlarson/elasticsearch-dsl-py#1 we've removed the Mapping function and added a fix for failing imports.

Can I take up this task?

Create an example FastAPI application with AsyncDocument and AsyncFacetedSearch

What should this app do? Do we have a sync equivalent of this?

clmnin avatar Jan 23 '21 17:01 clmnin

💚 CLA has been signed

@clmno You certainly can, I'll create individual issues for tasks that don't need to be finished in this PR so you can work in tandem.

Also, can you sign the CLA? Much appreciated!

sethmlarson avatar Jan 25 '21 13:01 sethmlarson

@seth-goodwin I've signed the CLA

clmnin avatar Jan 25 '21 13:01 clmnin

@sethmlarson How would I go about installing this locally and test it out?

pip3 install git+https://github.com/sethmlarson/elasticsearch-dsl-py.git@async-io

When I run this, the Async classes do not seem to be loaded.

zikphil avatar Feb 22 '21 16:02 zikphil

Would love to know what @zikphil asked as well!

MerlijnElderhuis avatar Feb 25 '21 13:02 MerlijnElderhuis

@sethmlarson I tried using AsyncSearh with next code:

from elasticsearch_dsl import AsyncSearch

search = AsyncSearch(using='my_connection_name', index='my_index_name')
search = search.filter('term', docType='my_docType_name')
result = search.count()

and I get the following error:

  <...>
  File "<..>/elasticsearch_dsl/_async/search.py", line 430, in count
    return await es.count(index=self._index, body=d, **self._params)["count"]
TypeError: 'coroutine' object is not subscriptable

I fix this the next code:

count = await es.count(index=self._index, body=d, **self._params)
return count["count"]

Proposal: fix this error similar code

maybe later I prepare PR

Edit: ready https://github.com/sethmlarson/elasticsearch-dsl-py/pull/2

jellex avatar Mar 23 '21 09:03 jellex

Any chance this can be merged?

nicholasamorim avatar Apr 29 '21 09:04 nicholasamorim

Any update on this?

jamesrappazzo avatar Sep 20 '21 20:09 jamesrappazzo

When this AsyncSearch feature will be availble.. I don't see part of 7.4 release

bizd-sk avatar Sep 28 '21 13:09 bizd-sk

Sorry all, I've been focused on work in the client lately so this work has been put on hold.

sethmlarson avatar Sep 28 '21 13:09 sethmlarson

@sethmlarson thanks for the hard work. Do you think much has remained for this PR? Any on guess when it would be available?

milani avatar Nov 23 '21 06:11 milani

@sethmlarson Hello, i am interested too in this functionality, any update on when it may be available? Is there anything i can do to help? Thank you :)

rocco8620 avatar Feb 04 '22 08:02 rocco8620

Hi, I am very interested in this functionality. May I ask why we don't merge this feature, please? Any update, please? Many thanks

nxhuy-github avatar Feb 28 '23 10:02 nxhuy-github

hey team, thank you so much for all the efforts. Could you please tell us more when do you expect to make it public?

dima-kov avatar Jul 31 '23 20:07 dima-kov