deepdiff
deepdiff copied to clipboard
Published
20 hours ago •
seperman
seperman
8.1.0
- Removing deprecated lines from setup.py
- Fixes
__slots__comparison when the attribute doesn't exist. - Relaxing orderly-set reqs
Codecov Report
Attention: Patch coverage is 93.93939% with 2 lines in your changes missing coverage. Please review.
Project coverage is 96.52%. Comparing base (
6d8a4c7) to head (32d60a9).
| Files with missing lines | Patch % | Lines |
|---|---|---|
| deepdiff/deephash.py | 66.66% | 2 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## master #483 +/- ##
==========================================
- Coverage 96.70% 96.52% -0.19%
==========================================
Files 14 14
Lines 3946 3971 +25
==========================================
+ Hits 3816 3833 +17
- Misses 130 138 +8
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi @artemisart Thanks for reviewing! Sorry I was too busy. I'm going to make a release over the weekend.
Hello!
I'm patching this branch in Debian to have deepdiff workin in python3.13. I have these tests failing. I'm didn't check in depth if is an error on my setup, or perhaps is something that can help you in something.
=================================== FAILURES ===================================
___________ TestDeltaCompareFunc.test_list_of_alphabet_and_its_delta ___________
self = <tests.test_delta.TestDeltaCompareFunc object at 0x7f83493da1e0>
def test_list_of_alphabet_and_its_delta(self):
l1 = "A B C D E F G D H".split()
l2 = "B C X D H Y Z".split()
diff = DeepDiff(l1, l2)
# Problem: The index of values_changed should be either all for AFTER removals or BEFORE removals.
# What we have here is that F & G transformation to Y and Z is not compatible with A and E removal
# it is really meant for the removals to happen first, and then have indexes in L2 for values changing
# rather than indexes in L1. Here what we need to have is:
# A B C D E F G D H
# A B C-X-E
# B C D F G D H # removal
# What we really need is to report is as it is in difflib for delta specifically:
# A B C D E F G D H
# B C D E F G D H delete t1[0:1] --> t2[0:0] ['A'] --> []
# B C D E F G D H equal t1[1:3] --> t2[0:2] ['B', 'C'] --> ['B', 'C']
# B C X D H replace t1[3:7] --> t2[2:3] ['D', 'E', 'F', 'G'] --> ['X']
# B C X D H equal t1[7:9] --> t2[3:5] ['D', 'H'] --> ['D', 'H']
# B C X D H Y Z insert t1[9:9] --> t2[5:7] [] --> ['Y', 'Z']
# So in this case, it needs to also include information about what stays equal in the delta
# NOTE: the problem is that these operations need to be performed in a specific order.
# DeepDiff removes that order and just buckets all insertions vs. replace vs. delete in their own buckets.
# For times that we use Difflib, we may want to keep the information for the array_change key
# just for the sake of delta, but not for reporting in deepdiff itself.
# that way we can re-apply the changes as they were reported in delta.
delta = Delta(diff)
assert l2 == l1 + delta
with pytest.raises(ValueError) as exc_info:
l1 == l2 - delta
assert "Please recreate the delta with bidirectional=True" == str(exc_info.value)
delta2 = Delta(diff, bidirectional=True)
assert l2 == l1 + delta2
assert l1 == l2 - delta2
dump = Delta(diff, bidirectional=True).dumps()
delta3 = Delta(dump, bidirectional=True)
assert l2 == l1 + delta3
assert l1 == l2 - delta3
dump4 = Delta(diff, bidirectional=True, serializer=json_dumps).dumps()
> delta4 = Delta(dump4, bidirectional=True, deserializer=json_loads)
tests/test_delta.py:2440:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
deepdiff/delta.py:130: in __init__
self.diff = _deserializer(diff, safe_to_import=safe_to_import)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
obj = '{"values_changed": {"root[3]": {"new_value": "X", "old_value": "D", "new_path": "root[2]"}, "root[5]": {"new_value": ...3, 7, 2, 3, ["D", "E", "F", "G"], ["X"]], ["equal", 7, 9, 3, 5, null, null], ["insert", 9, 9, 5, 7, [], ["Y", "Z"]]]}}'
safe_to_import = None
def _deserializer(obj, safe_to_import=None):
result = deserializer(obj)
if result.get('_iterable_opcodes'):
_iterable_opcodes = {}
for path, op_codes in result['_iterable_opcodes'].items():
_iterable_opcodes[path] = []
for op_code in op_codes:
_iterable_opcodes[path].append(
> Opcode(
**op_code
)
)
E TypeError: deepdiff.helper.Opcode() argument after ** must be a mapping, not list
deepdiff/delta.py:102: TypeError
___ TestDeepDiffText.test_exclude_path_when_prefix_of_exclude_path_matches1 ____
self = <tests.test_diff_text.TestDeepDiffText object at 0x7f834934f440>
def test_exclude_path_when_prefix_of_exclude_path_matches1(self):
diff = DeepDiff({}, {'foo': '', 'bar': ''}, exclude_paths=['foo', 'bar'])
> assert not diff
E AssertionError: assert not {'values_changed': {'root': {'new_value': {'foo': '', 'bar': ''}, 'old_value': {}}}}
tests/test_diff_text.py:1555: AssertionError
_____________________ TestDeepDiffText.test_bad_attribute ______________________
self = <tests.test_diff_text.TestDeepDiffText object at 0x7f8349355c70>
def test_bad_attribute(self):
class Bad:
__slots__ = ['x', 'y']
def __getattr__(self, key):
raise AttributeError("Bad item")
def __str__(self):
return "Bad Object"
t1 = Bad()
t2 = Bad()
ddiff = DeepDiff(t1, t2)
result = {'unprocessed': ['root: Bad Object and Bad Object']}
> assert result == ddiff
E AssertionError: assert {'unprocessed... Bad Object']} == {}
E
E Left contains 1 more item:
E {'unprocessed': ['root: Bad Object and Bad Object']}
E Use -v to get more diff
tests/test_diff_text.py:1791: AssertionError
_________ TestDeepDiffText.test_group_by_with_none_key_and_ignore_case _________
self = <tests.test_diff_text.TestDeepDiffText object at 0x7f8349356570>
def test_group_by_with_none_key_and_ignore_case(self):
"""Test that group_by works with None keys when ignore_string_case is True"""
dict1 = [{'txt_field': 'FULL_NONE', 'group_id': None}, {'txt_field': 'FULL', 'group_id': 'a'}]
dict2 = [{'txt_field': 'PARTIAL_NONE', 'group_id': None}, {'txt_field': 'PARTIAL', 'group_id': 'a'}]
> diff = DeepDiff(
dict1,
dict2,
ignore_order=True,
group_by='group_id',
ignore_string_case=True
)
tests/test_diff_text.py:2230:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
deepdiff/diff.py:332: in __init__
self._diff(root, parents_ids=frozenset({id(t1)}), _original_type=_original_type)
deepdiff/diff.py:1669: in _diff
self._diff_dict(level, parents_ids, local_tree=local_tree)
deepdiff/diff.py:605: in _diff_dict
t1_clean_to_keys = self._get_clean_to_keys_mapping(keys=t1_keys, level=level)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = {}, keys = [None, 'a']
level = <root t1:{None: {'txt...}, t2:{None: {'txt...}>
def _get_clean_to_keys_mapping(self, keys, level):
"""
Get a dictionary of cleaned value of keys to the keys themselves.
This is mainly used to transform the keys when the type changes of keys should be ignored.
TODO: needs also some key conversion for groups of types other than the built-in strings and numbers.
"""
result = dict_()
for key in keys:
if self.ignore_string_type_changes and isinstance(key, bytes):
clean_key = key.decode('utf-8')
elif self.use_enum_value and isinstance(key, Enum):
clean_key = key.value
elif isinstance(key, numbers):
type_ = "number" if self.ignore_numeric_type_changes else key.__class__.__name__
clean_key = self.number_to_string(key, significant_digits=self.significant_digits,
number_format_notation=self.number_format_notation)
clean_key = KEY_TO_VAL_STR.format(type_, clean_key)
else:
clean_key = key
if self.ignore_string_case:
> clean_key = clean_key.lower()
E AttributeError: 'NoneType' object has no attribute 'lower'
deepdiff/diff.py:560: AttributeError
___________ TestDeepDiffText.test_affected_root_keys_when_dict_empty ___________
self = <tests.test_diff_text.TestDeepDiffText object at 0x7f8349356750>
def test_affected_root_keys_when_dict_empty(self):
diff = DeepDiff({}, {1:1, 2:2}, threshold_to_diff_deeper=0)
assert [1, 2] == diff.affected_root_keys
diff2 = DeepDiff({}, {1:1, 2:2})
> assert [] == diff2.affected_root_keys
E assert [] == [not present]
E
E Right contains one more item: not present
E Use -v to get more diff
tests/test_diff_text.py:2251: AssertionError
_________________________ TestDeepHashPrep.test_polars _________________________
self = <tests.test_hash.TestDeepHashPrep object at 0x7f8349367530>
def test_polars(self):
> import polars as pl
E ModuleNotFoundError: No module named 'polars'
tests/test_hash.py:795: ModuleNotFoundError
_________ TestSerialization.test_serialization_text_force_builtin_json _________
self = <tests.test_serialization.TestSerialization object at 0x7f83489e9d90>
def test_serialization_text_force_builtin_json(self):
ddiff = DeepDiff(t1, t2)
> with pytest.raises(TypeError) as excinfo:
E Failed: DID NOT RAISE <class 'TypeError'>
tests/test_serialization.py:52: Failed
_______________ TestDeepDiffPretty.test_namedtuple_seriazliation _______________
self = <tests.test_serialization.TestDeepDiffPretty object at 0x7f8348a291c0>
def test_namedtuple_seriazliation(self):
op_code = Opcode(tag="replace", t1_from_index=0, t1_to_index=1, t2_from_index=10, t2_to_index=20)
serialized = json_dumps(op_code)
expected = '{"tag":"replace","t1_from_index":0,"t1_to_index":1,"t2_from_index":10,"t2_to_index":20,"old_values":null,"new_values":null}'
> assert serialized == expected
E assert '["replace", ..., null, null]' == '{"tag":"repl...values":null}'
E
E - {"tag":"replace","t1_from_index":0,"t1_to_index":1,"t2_from_index":10,"t2_to_index":20,"old_values":null,"new_values":null}
E + ["replace", 0, 1, 10, 20, null, null]
tests/test_serialization.py:412: AssertionError
____________________ TestDeepDiffPretty.test_reversed_list _____________________
self = <tests.test_serialization.TestDeepDiffPretty object at 0x7f8348a28530>
def test_reversed_list(self):
items = reversed([1, 2, 3])
serialized = json_dumps(items)
serialized2 = json_dumps(items)
> assert '[3,2,1]' == serialized
E AssertionError: assert '[3,2,1]' == '[3, 2, 1]'
E
E - [3, 2, 1]
E ? - -
E + [3,2,1]
tests/test_serialization.py:420: AssertionError
=============================== warnings summary ===============================
tests/test_serialization.py:391
/build/reproducible-path/deepdiff-8.0.1/.pybuild/cpython3_3.12_deepdiff/build/tests/test_serialization.py:391: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
(7, datetime.datetime.utcnow(), datetime.datetime.fromisoformat),
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
------------------------------------------------------------------------------------------------ benchmark: 3 tests ------------------------------------------------------------------------------------------------
Name (time in us) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_lfu[items0-3-expected_results0-1.333] 14.6970 (1.0) 313.0990 (2.98) 16.1242 (1.0) 6.5514 (1.27) 15.5090 (1.0) 0.5010 (1.0) 92;334 62.0184 (1.0) 7977 1
test_lfu[items1-3-expected_results1-1.666] 17.5230 (1.19) 105.0680 (1.0) 19.5968 (1.22) 5.1401 (1.0) 18.8250 (1.21) 1.3930 (2.78) 287;366 51.0287 (0.82) 25482 1
test_lfu[items2-3-expected_results2-3.333] 23.6950 (1.61) 161.3240 (1.54) 25.4352 (1.58) 5.6073 (1.09) 24.8160 (1.60) 0.5810 (1.16) 247;987 39.3155 (0.63) 23376 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
=========================== short test summary info ============================
FAILED tests/test_delta.py::TestDeltaCompareFunc::test_list_of_alphabet_and_its_delta
FAILED tests/test_diff_text.py::TestDeepDiffText::test_exclude_path_when_prefix_of_exclude_path_matches1
FAILED tests/test_diff_text.py::TestDeepDiffText::test_bad_attribute - Assert...
FAILED tests/test_diff_text.py::TestDeepDiffText::test_group_by_with_none_key_and_ignore_case
FAILED tests/test_diff_text.py::TestDeepDiffText::test_affected_root_keys_when_dict_empty
FAILED tests/test_hash.py::TestDeepHashPrep::test_polars - ModuleNotFoundErro...
FAILED tests/test_serialization.py::TestSerialization::test_serialization_text_force_builtin_json
FAILED tests/test_serialization.py::TestDeepDiffPretty::test_namedtuple_seriazliation
FAILED tests/test_serialization.py::TestDeepDiffPretty::test_reversed_list - ...
============= 9 failed, 937 passed, 10 skipped, 1 warning in 4.74s =============
Please ignore the polars package error. Any help or clue is welcome :)
Hi @eamanu All these tests pass for me on Ubuntu 24.04.1 and Python 3.13. What version of Debian are you using?
Hi @seperman
Thanks for your response. I'm in Debian sid. Now, as you merged I will wait the release to package it.
I suspect that I missed something while patching the PR.
Hi @seperman,
The issue was because orjson is a dependency. In despite that deepdiff has a import error manage, the tests need it, if not they fail.
Hi @eamanu Yeah, you need to install the requirements-dev.txt before running the tests. You can take a look at the github actions that run the tests.