elasticsearch-py icon indicating copy to clipboard operation
elasticsearch-py copied to clipboard

Log scroll response failures

Open bartier opened this issue 4 years ago • 6 comments

As raised by issue #1261, when the endpoint POST /_search/scroll answers with failures indications they are not provided in the exception or log to help troubleshoot the root cause.

This PR intends to solve this adding failures indications like the example below: image

A reason limit (5) is set to avoid the exception end up massive as pointed by @tommyzli

bartier avatar Jul 07 '20 22:07 bartier

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

elasticmachine avatar Jul 07 '20 22:07 elasticmachine

@sethmlarson Hi! If possible, could you do a code review at this PR? :)

I guess me and @tommyzli agree on the proposal code changes already.

bartier avatar Jul 09 '20 22:07 bartier

@sethmlarson Sure. Unfortunately I'm struggling to run the test module where I want to add my tests. I made the following steps to run and it shows a error like 'AttributeError: 'TestXXXXX' object has no attribute 'client'. I guess somehow I am not able to start ES but I'm not sure how I should make this works. If I could make this work I could continue to work doing the tests.

Any idea if I'm doing something wrong?

Notes: I have installed docker/docker-compose and get_client from test_elasticsearch/test_server/__init__.py is raising error trying to import test_elasticsearch.local and also the fallback test.get_test_client(**kwargs) raises an error. image

$ git clone https://github.com/bartier/elasticsearch-py.git
$ cd elasticsearch-py && git checkout log-scroll-failures
$ python3.8 -m venv venv
$ source venv/bin/activate
$ python setup.py test -s test_elasticsearch.test_server # error happens here

# Below output was truncated
======================================================================
ERROR: test_all_documents_are_read (test_elasticsearch.test_server.test_helpers.TestScan)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vitor/github/elasticsearch-py/test_elasticsearch/test_server/test_helpers.py", line 362, in test_all_documents_are_read
    self.client.bulk(bulk, refresh=True)
AttributeError: 'TestScan' object has no attribute 'client'

======================================================================
ERROR: test_clear_scroll (test_elasticsearch.test_server.test_helpers.TestScan)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vitor/github/elasticsearch-py/test_elasticsearch/test_server/test_helpers.py", line 481, in test_clear_scroll
    self.client.bulk(bulk, refresh=True)
AttributeError: 'TestScan' object has no attribute 'client'

======================================================================  

bartier avatar Aug 21 '20 02:08 bartier

Hi! @sethmlarson @bartier Has there been any progress on this? I am running into The same error.

ajosh0504 avatar Feb 18 '21 23:02 ajosh0504

Hi! @sethmlarson @bartier Has there been any progress on this? I am running into The same error.

Unfortunately it has not been progress yet. I guess maintainers should have a look at this to give a hint at least how we could run the tests, so we can make unit tests and go ahead with PRs.

bartier avatar Feb 18 '21 23:02 bartier

Hi! @sethmlarson @bartier Has there been any progress on this? I am running into The same error.

Unfortunately it has not been progress yet. I guess maintainers should have a look at this to give a hint at least how we could run the tests, so we can make unit tests and go ahead with PRs.

Try to run tests over py.test command. e.g. $ py.test test_elasticsearch/test_server/test_helpers.py

Sparkycz avatar Mar 29 '21 14:03 Sparkycz