deepdiff
deepdiff copied to clipboard
seperman
RecursionError
Please checkout the F.A.Q page before creating a bug ticket to make sure it is not already addressed. Searched FAQ for issues containing word "Recursion" but found none.
Describe the bug RecursionError: maximum recursion depth exceeded while calling a Python object
To Reproduce I use the Robot Framwork library DocTestLibrary to compare PDF's. So I hava a PDF with expected result and I have the actual PDF from the testrun. There is a keyword "Compare Pdf Documents" which compares there 2 PDF's. Somewhere mid-May this keyword produces the error "RecursionError: maximum recursion depth exceeded while calling a Python object", but the test was still the same. We found that de deepdiff-library changed mid-May from version 5.8.0 to 5.8.1. If I explicitely make a docker-image with version 5.8.0 I get that the test PASSED. If I expliciteyl make a docker-image with verison 5.8.1 I get the recursion-error.
Expected behavior I do not expect a recursion-error, but I expect the test to PASS.
OS, DeepDiff version and Python version (please complete the following information):
- OS: linux (docker-image with platform linux).
- PYTHON_VERSION=3.10.3
- Version Deepdiff 5.8.0 and 5.8.1
- robotframework-doctestlibrary version 0.2.0.20220325161430
Additional context Snippet of debug: KEYWORD keywords . En wordt het gedownloade bestand gecheckt op correctheid (overzicht) Start / End / Elapsed: 20220524 10:05:48.458 / 20220524 10:05:49.785 / 00:00:01.327 00:00:00.000KEYWORD ${TESTEXECDIR} = BuiltIn . Set Variable If '${BROWSERCHOICE}' == 'ChromeLocalDownload', ${EXECDIR}/klicviewer/resources/printen/overzicht/ Documentation: Sets variable based on the given condition.
Start / End / Elapsed: 20220524 10:05:48.458 / 20220524 10:05:48.458 / 00:00:00.000 10:05:48.458 INFO ${TESTEXECDIR} = /var/jenkins_data/workspace/klic-win-test-automation/klicviewer/resources/printen/overzicht/ 00:00:01.325IF '${TEST NAME}'=='${TEST NAME}' Start / End / Elapsed: 20220524 10:05:48.459 / 20220524 10:05:49.784 / 00:00:01.325 00:00:01.325KEYWORD DocTest.PdfTest . Compare Pdf Documents ${TESTEXECDIR}${TEST NAME}.pdf, ${download directory}/${TEST NAME}.pdf, compare=text | fonts | images | signatures Documentation: Compares some PDF metadata/properties of reference_document and candidate_document.
Start / End / Elapsed: 20220524 10:05:48.459 / 20220524 10:05:49.784 / 00:00:01.325 10:05:49.772 FAIL RecursionError: maximum recursion depth exceeded while calling a Python object 10:05:49.780 DEBUG Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/DocTest/PdfTest.py", line 107, in compare_pdf_documents diff = DeepDiff(ref_page['mediabox'], cand_page['mediabox']) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 296, in init self._diff(root, parents_ids=frozenset({id(t1)}), _original_type=_original_type) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids)
Hi, it is difficult to debug it without have reproducible code.
If you can share the portion of data that causes the issue, I can take a look at it.
Sep Dehpour
On May 27, 2022, at 12:29 AM, kad-buitea1 @.***> wrote:
Please checkout the F.A.Q page before creating a bug ticket to make sure it is not already addressed. Searched FAQ for issues containing word "Recursion" but found none.
Describe the bug RecursionError: maximum recursion depth exceeded while calling a Python object
To Reproduce I use the Robot Framwork library DocTestLibrary to compare PDF's. So I hava a PDF with expected result and I have the actual PDF from the testrun. There is a keyword "Compare Pdf Documents" which compares there 2 PDF's. Somewhere mid-May this keyword produces the error "RecursionError: maximum recursion depth exceeded while calling a Python object", but the test was still the same. We found that de deepdiff-library changed mid-May from version 5.8.0 to 5.8.1. If I explicitely make a docker-image with version 5.8.0 I get that the test PASSED. If I expliciteyl make a docker-image with verison 5.8.1 I get the recursion-error.
Expected behavior I do not expect a recursion-error, but I expect the test to PASS.
OS, DeepDiff version and Python version (please complete the following information):
OS: linux (docker-image with platform linux). PYTHON_VERSION=3.10.3 Version Deepdiff 5.8.0 and 5.8.1 robotframework-doctestlibrary version 0.2.0.20220325161430 Additional context Snippet of debug: KEYWORD keywords . En wordt het gedownloade bestand gecheckt op correctheid (overzicht) Start / End / Elapsed: 20220524 10:05:48.458 / 20220524 10:05:49.785 / 00:00:01.327 00:00:00.000KEYWORD ${TESTEXECDIR} = BuiltIn . Set Variable If '${BROWSERCHOICE}' == 'ChromeLocalDownload', ${EXECDIR}/klicviewer/resources/printen/overzicht/ Documentation: Sets variable based on the given condition.
Start / End / Elapsed: 20220524 10:05:48.458 / 20220524 10:05:48.458 / 00:00:00.000 10:05:48.458 INFO ${TESTEXECDIR} = /var/jenkins_data/workspace/klic-win-test-automation/klicviewer/resources/printen/overzicht/ 00:00:01.325IF '${TEST NAME}'=='${TEST NAME}' Start / End / Elapsed: 20220524 10:05:48.459 / 20220524 10:05:49.784 / 00:00:01.325 00:00:01.325KEYWORD DocTest.PdfTest . Compare Pdf Documents ${TESTEXECDIR}${TEST NAME}.pdf, ${download directory}/${TEST NAME}.pdf, compare=text | fonts | images | signatures Documentation: Compares some PDF metadata/properties of reference_document and candidate_document.
Start / End / Elapsed: 20220524 10:05:48.459 / 20220524 10:05:49.784 / 00:00:01.325 10:05:49.772 FAIL RecursionError: maximum recursion depth exceeded while calling a Python object 10:05:49.780 DEBUG Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/DocTest/PdfTest.py", line 107, in compare_pdf_documents diff = DeepDiff(ref_page['mediabox'], cand_page['mediabox']) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 296, in init self._diff(root, parents_ids=frozenset({id(t1)}), _original_type=_original_type) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids)
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.
Hi, I have a very large and complex object tree with deeply nested self-references. So, it's an endless tree. Therefore I also get recursion errors.
Maybe I missed it, but to my understanding there is no parameter to define when to stop comparison at a specific search/recursion depth, right? How could I achieve to break after a depth of e.g. 5?
max_passes is an option to overcome this limitation, but it does serve a different purpose and it does not solve the problem as the algorithm gets stuck in the first recursion it will trap into.
Idea: A cool option would be a parameter like ignore_recursion, that hinders the diff process to follow paths that have already been visited.
Thomas Very cool and helpful library 👍
Thanks, No max depth but you have max diffs for example:
https://zepworks.com/deepdiff/current/optimizations.html#
Sep Dehpour
On Jun 16, 2022, at 3:01 AM, Thomas Zeutschler @.***> wrote:
Hi, I have a very large and complex object tree with deeply nested and self-references. So, it's an endless tree.
Maybe I missed it, but to my understanding there is no parameter to define when to stop comparison at a specific search/recursion depth, right? How could I achieve to break ofter a depth of e.g. 5?
Thomas Very cool and helpful library 👍
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.
Hi,
I hope this will help. The language in which my testcases are written is in Dutch. So I’ll try to describe what happens. Basis of my environment is robotframework 5 and robotframework-browser 12.4.0. Also robotframework-doctestlibrary 0.2.0.20220325161430.
There are 2 testcases in klic_viewer_print_deepdiff_suite.robot. The first one should pass (just to check that whatever you tried works). The second testcase is the one that fails with deep-diff version 5.8.1 but passes with 5.8.0. The keyword / teststep where it fails is “En wordt het gedownloade bestand gecheckt op correctheid (overzicht)”. (which roughly translates to “And the downloaded document is checked for correctness (overview)”)
This keyword in full is this: ‘En wordt het gedownloade bestand gecheckt op correctheid (overzicht) ${TESTEXECDIR}= Set Variable If '${BROWSERCHOICE}' == 'ChromeLocalDownload' ... ${EXECDIR}/deepdiff/resources/printen/overzicht/ IF '${TEST NAME}'=='${TEST NAME}' Compare Pdf Documents ${TESTEXECDIR}${TEST NAME}.pdf ${download directory}/${TEST NAME}.pdf ... compare=text | fonts | images | signatures END’
The keyword ‘Compare Pdf Documents’ comes from the robotframework-doctestlibrary and that one calls the deepdiff-library.
The ‘actual’ / created in testcase .PDF is made in step ‘Dan wordt het bestand naar de juiste directory gedownload’. The expected .PDF is in the .zip file. And these two .PDF’s will be compared in the keyword ‘Compare Pdf Documents’.
I do not have the underlying code for that keyword.
Hopefully this will help. Just ask if you need anything else. I’ll try to help in any way I can.
With kind regards, Angela Buitenhuis.
Van: Sep Dehpour @.> Verzonden: vrijdag 27 mei 2022 16:36 Aan: seperman/deepdiff @.> CC: Buitenhuis, Angela @.>; Author @.> Onderwerp: Re: [seperman/deepdiff] RecursionError (Issue #322)
Hi, it is difficult to debug it without have reproducible code.
If you can share the portion of data that causes the issue, I can take a look at it.
Sep Dehpour
On May 27, 2022, at 12:29 AM, kad-buitea1 @.***mailto:***@***.***> wrote:
Please checkout the F.A.Q page before creating a bug ticket to make sure it is not already addressed. Searched FAQ for issues containing word "Recursion" but found none.
Describe the bug RecursionError: maximum recursion depth exceeded while calling a Python object
To Reproduce I use the Robot Framwork library DocTestLibrary to compare PDF's. So I hava a PDF with expected result and I have the actual PDF from the testrun. There is a keyword "Compare Pdf Documents" which compares there 2 PDF's. Somewhere mid-May this keyword produces the error "RecursionError: maximum recursion depth exceeded while calling a Python object", but the test was still the same. We found that de deepdiff-library changed mid-May from version 5.8.0 to 5.8.1. If I explicitely make a docker-image with version 5.8.0 I get that the test PASSED. If I expliciteyl make a docker-image with verison 5.8.1 I get the recursion-error.
Expected behavior I do not expect a recursion-error, but I expect the test to PASS.
OS, DeepDiff version and Python version (please complete the following information):
OS: linux (docker-image with platform linux). PYTHON_VERSION=3.10.3 Version Deepdiff 5.8.0 and 5.8.1 robotframework-doctestlibrary version 0.2.0.20220325161430 Additional context Snippet of debug: KEYWORD keywords . En wordt het gedownloade bestand gecheckt op correctheid (overzicht) Start / End / Elapsed: 20220524 10:05:48.458 / 20220524 10:05:49.785 / 00:00:01.327 00:00:00.000KEYWORD ${TESTEXECDIR} = BuiltIn . Set Variable If '${BROWSERCHOICE}' == 'ChromeLocalDownload', ${EXECDIR}/klicviewer/resources/printen/overzicht/ Documentation: Sets variable based on the given condition.
Start / End / Elapsed: 20220524 10:05:48.458 / 20220524 10:05:48.458 / 00:00:00.000 10:05:48.458 INFO ${TESTEXECDIR} = /var/jenkins_data/workspace/klic-win-test-automation/klicviewer/resources/printen/overzicht/ 00:00:01.325IF '${TEST NAME}'=='${TEST NAME}' Start / End / Elapsed: 20220524 10:05:48.459 / 20220524 10:05:49.784 / 00:00:01.325 00:00:01.325KEYWORD DocTest.PdfTest . Compare Pdf Documents ${TESTEXECDIR}${TEST NAME}.pdf, ${download directory}/${TEST NAME}.pdf, compare=text | fonts | images | signatures Documentation: Compares some PDF metadata/properties of reference_document and candidate_document.
Start / End / Elapsed: 20220524 10:05:48.459 / 20220524 10:05:49.784 / 00:00:01.325 10:05:49.772 FAIL RecursionError: maximum recursion depth exceeded while calling a Python object 10:05:49.780 DEBUG Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/DocTest/PdfTest.py", line 107, in compare_pdf_documents diff = DeepDiff(ref_page['mediabox'], cand_page['mediabox']) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 296, in init self._diff(root, parents_ids=frozenset({id(t1)}), _original_type=_original_type) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 407, in _diff_obj self._diff_dict( File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 547, in _diff_dict self._diff(next_level, parents_ids_added) File "/usr/local/lib/python3.10/site-packages/deepdiff/diff.py", line 1354, in _diff self._diff_obj(level, parents_ids)
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.
— Reply to this email directly, view it on GitHubhttps://github.com/seperman/deepdiff/issues/322#issuecomment-1139679566, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOPVJIR6SN4APODFXMX7W5LVMDMVJANCNFSM5XDMKUJA. You are receiving this because you authored the thread.Message ID: @.@.>>
Disclaimer: De inhoud van deze e-mail is vertrouwelijk en uitsluitend bestemd voor de geadresseerde(n). Gebruik, openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is niet toegestaan. Op al onze producten en diensten zijn onze algemene leveringsvoorwaarden van toepassing [https://www.kadaster.nl/algemene-leveringsvoorwaarden].
Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Our general terms and conditions of delivery apply to all our products and services [https://www.kadaster.com/general-terms-and-conditions].
Hi @Zeutschler and @kad-buitea1 This looks like a bug. DeepDiff should keep track of the "ID" of the objects it has already visited so even if there is a self reference, it should not get into the recursion. It seems like in this case there is a bug where somewhere we don't keep track of an object. But in order for me to debug this, I need to be able to reproduce it. If you look at our test cases, we already have tests for recursion: https://github.com/seperman/deepdiff/blob/master/tests/test_diff_text.py#L840-L938
If you can drop a data sample here that causes the issue, I can debug it.
Hello @Zeutschler and @kad-buitea1! One year has passed. If you can please share reproducible code, that would be great. I can't fix it until I can reproduce it.
Hi @seperman I've updated to the newest version of robotframework-doctestlibrary (0.19..0) and the problems is solved in that version (for me). Thank you for your help.