neomodel icon indicating copy to clipboard operation
neomodel copied to clipboard

concurrent async calls of `get_or_create` lead to duplicate nodes

Open AnikinNN opened this issue 9 months ago • 1 comments

Expected Behavior (Mandatory)

At concurrent calling of get_or_create from different places of a program we get the same node which was created once

Actual Behavior (Mandatory)

At concurrent calling of get_or_create from different places of a program we can get different nodes which we consider as duplicates

How to Reproduce the Problem

Run the code in example. Probably you should increase the amount of created nodes if your's hardware better than mine. I wrapped code into for loop because sometimes the bug can not be caught on the first attempt. Looks like race condition.

Simple Example

As a simple alternative to concurrent calls from different places of entire program we can use asyncio.gather

import asyncio

import neomodel.config
from neomodel import AsyncStructuredNode, StringProperty

neomodel.config.DATABASE_URL = 'bolt://neo4j:12345678@neo4j:7687'
name = 'some_name'


class SomeNode(AsyncStructuredNode):
    name = StringProperty(required=True)


async def main():
    for counter in range(100):
        print(f'iteration {counter}')

        # cleanup
        await neomodel.adb.cypher_query("MATCH (n) DETACH DELETE n")

        # create a lot of the same nodes
        # this should return a lot of instances with the same id
        created_nodes = await asyncio.gather(*(
            SomeNode.get_or_create(dict(name=name))
            for _ in range(1000)
        ))

        # check that all instances have the same id
        ids = set(i[0].element_id_property for i in created_nodes)
        assert len(ids) == 1

        # check that there is only one node in the neo4j
        assert len(await SomeNode.nodes) == 1


if __name__ == '__main__':
    asyncio.run(main())

Specifications (Mandatory)

Currently used versions

Versions

-OS: linux, 3.10.14-slim -Library:5.3.0 [extras] -Neo4j:5.19.0 community

AnikinNN avatar May 30 '24 14:05 AnikinNN