Question about LOCOMO benchmark results inconsistency with mem0 paper

[wangbiqiang-mt]

Thank you for your contributions to the memory graph field. However, I have some questions regarding the experimental results on the LOCOMO benchmark (from https://www.getzep.com/mem0-alternative/) that need clarification.

1. Why do the experimental results reported here show significant inconsistencies with the experimental results in the mem0 paper?

[danielchalef]

@wangbiqiang-mt We wrote about why Mem0's evaluation was inaccurate here: https://blog.getzep.com/lies-damn-lies-statistics-is-mem0-really-sota-in-agent-memory/

Hope this is helpful!

[wangbiqiang-mt]

@danielchalef Got it! By the way, I was wondering if there are evaluation results for each category in the LOCOMO dataset (Single Hop, Multi Hop, Temporal, and Open Domain). This might be helpful for comparison in subsequent work evaluations.