assert-json-diff icon indicating copy to clipboard operation
assert-json-diff copied to clipboard

avoid recursion

Open yanns opened this issue 2 years ago • 12 comments

When comparing deep json structures, the recursion can be dangerous.

yanns avatar Jun 29 '22 18:06 yanns

As far as I understand recursion is mainly dangerous because you risk overflowing the stack. However I think thats unlikely when diffing JSON, unless your JSON has huge levels of nestings. What do you think?

Have you seen any problems caused by the recursion or have you benchmarked it?

davidpdrsn avatar Jun 29 '22 21:06 davidpdrsn

I have not seen any overflow, so I think we're safe here.

What I can observe on production: when we use this change + https://github.com/davidpdrsn/assert-json-diff/pull/30, then the server consumes much less memory.

yanns avatar Jun 30 '22 08:06 yanns

Hm I wouldn't expect recursion to impact memory usage that much. Have you tried it with just https://github.com/davidpdrsn/assert-json-diff/pull/30?

davidpdrsn avatar Jun 30 '22 09:06 davidpdrsn

Hm I wouldn't expect recursion to impact memory usage that much. Have you tried it with just #30?

No I have not. This would take some time. Do you need that?

yanns avatar Jun 30 '22 11:06 yanns

No I have not. This would take some time. Do you need that?

Yes. I don't think we refactor the code without a good reason as the change might introduce bugs.

davidpdrsn avatar Jun 30 '22 11:06 davidpdrsn

No I have not. This would take some time. Do you need that?

Yes. I don't think we refactor the code without a good reason as the change might introduce bugs.

OK I did the experiment. The memory usage went from 120 MB to 320 MB.

yanns avatar Jun 30 '22 14:06 yanns

Alright. Thanks for testing it! I'll actually review the code then :)

davidpdrsn avatar Jun 30 '22 14:06 davidpdrsn

cargo clippy is failing for parts of the code that were not touched by this PR. If you want, I can open a new PR for that.

yanns avatar Jun 30 '22 15:06 yanns

I wont say no to that. Go for it! 😅

davidpdrsn avatar Jun 30 '22 15:06 davidpdrsn

https://github.com/davidpdrsn/assert-json-diff/pull/32

yanns avatar Jun 30 '22 15:06 yanns

@davidpdrsn The pipeline is green now.

yanns avatar Aug 24 '22 12:08 yanns

@davidpdrsn ok for you to merge this?

yanns avatar Sep 05 '22 16:09 yanns