cartography icon indicating copy to clipboard operation
cartography copied to clipboard

Support configuring `max_connection_idle_time` in Neo4j driver

Open achantavy opened this issue 2 months ago • 2 comments

Support configuring max_connection_idle_time in Neo4j driver

Description

Currently, Cartography allows configuration of the Neo4j driver via parameters such as max_connection_lifetime and connection_timeout, but does not expose max_connection_idle_time. This setting was introduced in newer versions of the Neo4j Python driver to proactively close idle Bolt connections before the server does.

Supporting max_connection_idle_time in Cartography’s Neo4j driver initialization would let deployments avoid frequent SessionExpired or ConnectionResetError issues that occur when Aura or clustered Neo4j instances close idle connections.

The desired behavior is to allow users to configure this via the standard --neo4j-max-connection-idle-time CLI argument or environment variable, which then gets passed through to the GraphDatabase.driver() call.

Example:

driver = GraphDatabase.driver(
    uri,
    auth=auth,
    max_connection_lifetime=config.neo4j_max_connection_lifetime,
    max_connection_idle_time=config.neo4j_max_connection_idle_time,
)

Motivation

This feature helps prevent connection resets and SessionExpired errors that occur after several minutes of inactivity between stages in long-running sync jobs (e.g. during large GitHub ingestions).
Neo4j Aura and other managed clusters frequently close idle connections after ~3–5 minutes. Without this setting, the driver keeps stale sockets in its pool, which only fail on the next use, causing sync interruptions.

Allowing max_connection_idle_time aligns with Neo4j best practices and improves stability for large or multi-tenant syncs that may have uneven workloads across stages.

Alternatives Considered

  • Manual retry logic: Works but only recovers after failure
  • Refactoring to allow the Driver to be accessed throughout syncs instead of Session: very big and invasive refactor
  • Reducing max_connection_lifetime: Helps somewhat but does not address idle sockets specifically.

Adding max_connection_idle_time is the cleanest, officially supported solution.

Relevant Links

  • Neo4j Python driver docs: https://neo4j.com/docs/api/python-driver/current/api.html#neo4j.GraphDatabase.driver
  • Neo4j Aura recommendations on idle connections
  • Related Cartography issue: #1957

achantavy avatar Oct 07 '25 00:10 achantavy

Hi @achantavy I’d like to take this up.

From the discussion, the goal here is to expose the Neo4j driver’s max_connection_idle_time parameter through Cartography’s configuration layer — similar to how max_connection_lifetime and connection_timeout are handled.

I’ll add:

a new CLI argument --neo4j-max-connection-idle-time

a corresponding environment variable NEO4J_MAX_CONNECTION_IDLE_TIME, and

pass it through to the GraphDatabase.driver() initialization call.

This will allow users to configure how long idle connections are kept before being closed, helping prevent SessionExpired and ConnectionResetError issues on Neo4j Aura or clustered setups.

Can you please assign this issue under Hacktoberfest. Will be back with a PR soon after. Thanks

SK8-infi avatar Oct 27 '25 10:10 SK8-infi

i can't seem to find documentation for an neo4j official support for max_connection_idle_time anywhere. any pointers to which version of neo4j actually does have it ?? its neither in 6.0.2 nor is it in 5.28.2. i know we only support neo4j (>=5.28.2,<6.0.0) but i checked anyways...

iamsyedomair avatar Nov 06 '25 00:11 iamsyedomair